First, we would like to highlight that CBench is a test suite that could include all available benchmarks that are used to evaluate QA systems. We leave the choice of the benchmark(s) that the user would like to use for evaluation to the user. That said, it is a common practice in QA papers that older benchmarks are used and not necessarily the most recent. Part of the reason for this is to easily compare to previous systems. For example, the recent 2 QA systems: TemplateQA [Zheng et al., 2018] and KBQA [Cui et al., 2017] which published in VLDB used (QALD-5) and (QALD-5,QALD-3,QALD-1) respectively in their evaluation. So, including older benchmarks in CBench is a feature we believe the users will benefit from.
[Cui et al., 2017] Cui, W., Xiao, Y., Wang, H., Song, Y., Hwang, S.-w., and Wang, W. (2017).Kbqa: Learning question answering over qa corpora and knowledge bases.Proc.VLDBEndow., 10(5):565–576.
[Zheng et al., 2018] Zheng, W., Yu, J. X., Zou, L., and Cheng, H. (2018). Question answeringover knowledge graphs: question understanding via template decomposition.ProceedingsoftheVLDBEndowment, 11(11):1373–1386.