孙越凡,杨亮,林原,许侃,林鸿飞. 面向特定领域中文阅读理解数据集研究[J]. 中文信息学报, 2022, 36(12): 44-51.
SUN Yuefan, YANG Liang, LIN Yuan, XU Kan, LIN Hongfei. A Domain Specific Chinese Reading Comprehension Data Set. , 2022, 36(12): 44-51.
面向特定领域中文阅读理解数据集研究
孙越凡,杨亮,林原,许侃,林鸿飞
大连理工大学 信息检索研究室,辽宁 大连 116000
A Domain Specific Chinese Reading Comprehension Data Set
SUN Yuefan, YANG Liang, LIN Yuan, XU Kan, LIN Hongfei
Information Retrieval Laboratory, Dalian University of Technology, Liaoning, Dalian 116000, China
Abstract：This paper proposes a Chinese reading comprehension dataset-Restaurant(Res) for a specific field(catering industry). The data are collected from the Dianping application, with user reviews in the catering industry. The annotators provide questions and annotate the answers according to the date. There are currently two versions of the Res dataset: Res_v1 contains only questions with answers in user comments, and Res_v2 includes additional questions without answers in the comments. We apply the mainstream BiDAF, QANet and Bert models in the dataset, achieving as high as 73.78% accuracy. lagging far behind human performance of 91.03%.
[1] Chen D,Bolton J,Manning C D. A thorough examination of the CNN/daily mail reading comprehension task[C]//Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics,2016: 2358-2367. [2] Lai G,Xie Q,Liu H,et al. RACE: Large-scale ReAding comprehension dataset from examinations[C]//Proceedings of the Conference on Empirical Methods in Natural Language Processing,2017: 785-794. [3] Rajpurkar P,Zhang J,Lopyrev K,et al. SQuAD: 100,000+ questions for machine comprehension of text[C]//Proceedings of the Conference on Empirical Methods in Natural Language Processing,2016: 2383-2392. [4] Rajpurkar P,Jia R,Liang P. Know what you don't know: unanswerable questions for SQuAD[C]//Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics,2018: 784-789. [5] He W,Liu K,Liu J,etal. DuReader: A Chinese machine reading comprehension dataset from real-world applications[C]//Proceedings of the ACL Workshop on Vachine Reading for Ouestion Answerny,2018: 37-46. [6] Cui Y,Liu T,Che W,et al. A Span-extraction dataset for Chinese machine reading comprehension[C]//Proceedings of the Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing,2019: 5883-5889. [7] Wang B,Yao T,Zhang Q,et al.Reco: A large scale Chinese reading comprehension dataset on opinion[C]//Proceedings of the AAAI Conference on Artificial Intelligence,2020,34(05): 9146-9153. [8] Kenton J D M W C,Toutanova L K. BERT: Pre-training of deep bidirectional transformers for language understanding[C]//Proceedings of NAACL-HLT,2019: 4171-4186. [9] Reddy S,Chen D,Manning C D. CoQA: A conver sational question answering challenge[J]. Transactions of the Association for Computational Linguistics,2019,7: 249-266. [10] Nguyen T,Rosenberg M,Song X,et al. MS MARCO: A human generated machine reading comprehension dataset[J]. arXiv preprint arXiv: 1611.09268,2016. [11] Trischler A,Wang T,Yuan X,et al. NewsQA: A machine comprehension dataset[C]//Proceedings of 2nd Workshop on Representation Learning for NLP,2017: 191-200. [12] Xu H,Liu B,Shu L,et al. BERT Post-training for review reading comprehension and aspect-based sentiment analysis[C]//Proceedings of NAACL-HLT,2019: 2324-2335. [13] Xie Q,Lai G,Dai Z,et al. Large-scale cloze test dataset designed by teachers[J]. arXiv preprint arXiv: 1711.03225,2017. [14] Seo M,Kembhavi A,Farhadi A,et al. Bidirectional attention flow for machine comprehension[J]. arXiv preprint arXiv: 1611.01603,2016. [15] Yu W,Dohan D,Luong M T,et al. Combining local convolution with global self-attention for reading comprehension[J]. arXiv preprint arXiv: 1804.09541,2018.
2022, Vol. 36 
