机器阅读理解是基于给定文本,自动回答与文本内容相关的问题。针对此任务,学术界与工业界提出多个数据集与模型,促使阅读理解取得了一定的进步,但提出的模型大多只是针对某一类问题,不能满足现实世界问题多样性的需求。因此,该文针对阅读理解中问题类型多样性的解答展开研究,提出一种基于Bert的多任务阅读理解模型,利用注意力机制获得丰富的问题与篇章的表示,并对问题进行分类,然后将分类结果用于任务解答,实现问题的多样性解答。该文在中文公共阅读理解数据集CAIL2019-CJRC上对所提模型进行了实验,结果表明,系统取得了比所有基线模型都要好的效果。
Abstract
Machine reading comprehension (MRC) enables the machine read a given passage and then answer some relevant questions. A number of data sets and models have been proposed for a specific type of problems, without dealing with the diversity of problems in real-world. In this paper, we propose a multi-task reading comprehension model based on Bert. It uses the attention mechanism to obtain multi representations of questions and passages and then classify the questions. Then the model utilizes the classification results to answer the various questions. Experiments on Chinese public machine reading comprehension dataset CAIL2019-CJRC show that our system achieves better results than all the baseline models.
关键词
阅读理解 /
分类 /
注意力机制 /
多类型问题
{{custom_keyword}} /
Key words
reading comprehension /
classification /
attention mechanism /
multi-type questions
{{custom_keyword}} /
{{custom_sec.title}}
{{custom_sec.title}}
{{custom_sec.content}}
参考文献
[1] Minjoon Seo,Aniruddha Kembhavi, Ali Farhadi, et al. Bidirectional attention flow for machine comprehension[J]. arXiv preprint arXiv:1611.01603, 2016.
[2] Shuohang Wang,Jing Jiang. Machine compre hension using match-LSTM and answer pointer[J]. arXiv preprint arXiv:1608.07905, 2016.
[3] Jacob Devlin, Mingwei Chang, Kenton Lee,et al. BERT: Pre-training of deep bidirectional transformers for language under standing[J]. arXiv preprint arXiv:1810.04805, 2018.
[4] Yiming Cui, Ting Liu, Zhipeng Chen, et al. Consensus attention-based neural networks for chinese reading compre hension[C]//Proceedings of COLING 2016, the 26th International Conference on Computational Linguistics, 2016: 1777-1786.
[5] Karl Moritz Hermann,Tomas Kocisky,Edward Grefenstette, et al. Teaching machines to read and comprehend[C]//Proceedings of Advances in Neural Information Processing Systems, 2015:1693-1701.
[6] Guokun Lai, Qizhe Xie, Hanxiao Liu, et al. RACE: Large-scale reAding comprehension dataset from examinations[C]//Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, 2017:785-794.
[7] Matthew Richardson, J C Christopher Burges, Erin Renshaw.MCTest: A challenge dataset for the open-domain machine comprehension of text[C]//Proceedings of the Conference on Empirical Methods in Natural Language Processing, 2013:193-203.
[8] Pranav Rajpurkar, Jian Zhang, Konstantin Lopyrev, et al. SQuAD: 100,000+ questions for machine comprehension of text[C]//Proceedings of the Conference on Empirical Methods in Natural Language Processing, 2016: 2383-2392.
[9] Yiming Cui, Ting Liu, Wanxiang Che, et al. A span-extraction dataset for chinese machine reading comprehension[J]. arXiv preprint arXiv:1810.07366, 2018.
[10] Wei He, Kai Liu, Jing Liu, et al.DuReader: A Chinese machine reading comprehension dataset from real-world applications[J]. arXiv preprint arXiv: 1711.05073, 2017.
[11] Karl Moritz Hermann, Tomas Kocisky, Edward Grefenstette, et al. Teaching machines to read and comprehend[C]//Proceedings of Advances in Neural Information Processing Systems, 2015:1693-1701.
[12] Danqi Chen, Jason Bolton, Christopher D Manning. A thorough examination of the CNN/Daily Mail reading comprehension task[J]. arXiv preprint arXiv:1606.02858, 2016.
[13] Yiming Cui, Zhipeng Chen, Si Wei, et al. Attention-over attention neural networks for reading comprehension[J]. arXiv preprint arXiv:1607.04423, 2016.
[14] Wenhui Wang, Nan Yang, Furu Wei, et al. Gated self-matching networks for reading comprehension and question answering[C]//Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics, 2017:189-198.
[15] Zhipeng Chen, Yiming Cui, Wentao Ma, et al. Convolutional spatial attention model for reading comprehension with multiple-choice questions[J]. arXiv preprint arXiv: 1811.08610, 2018.
[16] Ashish Vaswani, Noam Shazeer, Niki Parmar, et al. Attention is all you need[J]. arXiv preprint arXiv:1706.3762, 2017.
[17] Yu Sun,Shuohuan Wang, Yukun Li, et al. ERNIE: Enhanced representation through knowledge integration[J]. arXiv preprint arXiv:1904.09223, 2019.
[18] Jimmy Lei Ba, Jamie Ryan Kiros, Geoffrey E Hinton. Layer normalization[J]. arXiv preprint arXiv:1607.06450, 2016.
[19] Yiming Cui, Wanxiang Che, Ting Liu, et al. Pre-training with whole word masking for chinese BERT[J]. arXiv preprint arXiv:1906.08101, 2019.
[20] Siva Reddy, Danqi Chen, Christopher D Manning. CoQA: A conversational question answering challenge[J]. arXiv preprint arXiv:1808.07042, 2018.
{{custom_fnGroup.title_cn}}
脚注
{{custom_fn.content}}
基金
国家重点研发计划重点专项项目(2018YFB1005103);国家自然科学基金(61673248);山西省研究生联合培养基地人才培养项目(2018JD02)
{{custom_fund}}
PDF(2378 KB)
