针对机器阅读理解中观点型问题的求解,提出一个端到端深度学习模型,使用Bi-GRU对文章和问题进行上下文语义编码,然后运用基于拼接、双线性、点乘和差集4种函数的注意力加上Query2Context和Context2Query两个方向注意力的融合算法获取文章和问题的综合语义信息,之后运用多层注意力转移推理机制不断聚焦,进一步获取更加准确的综合语义,最终将其与候选答案进行比较,选出正确答案。该模型在AIchallager2018观点型阅读理解中文测试数据集上准确率达到76.79%,性能超过基线系统。此外,该文尝试文章以句子序列作为输入表示进行答案求解,准确率达到78.48%,获得较好试验效果。
Abstract
In order to solve the opinion-problems of machine reading comprehension, an end-to-end deep learning model is proposed. In this paper, Bi-GRU is used to contextually encode passages and problems. And then four kinds of attentions, including the concatenated attention, the bilinear attention ,the element-wise dot attention and minus attention, are applied with the fusion of Query2Context and Context2Query attentions to obtain the comprehensive semantic information of the passage and the problem. This model further employs the multi-level attention transfer reasoning mechanism to obtain more accurate comprehensive semantics. The accuracy reaches 76.79% on the AIchallager 2018 opinion reading comprehension Chinese test data set. In addition, using the sentence sequence as input, the method could be boosted to an accuracy of 78.48%.
关键词
深度学习 /
机器阅读理解 /
注意力机制 /
Bi-GRU
{{custom_keyword}} /
Key words
deep learning /
machine reading comprehension /
attention mechanism /
Bi-GRU
{{custom_keyword}} /
{{custom_sec.title}}
{{custom_sec.title}}
{{custom_sec.content}}
参考文献
[1] 梁小波,任飞亮,刘永康,等. N-Reader: 基于双层Self-attention的机器阅读理解模型[J]. 中文信息学报, 2018, 32(10): 130-137.
[2] Richardson M,Burges C J C,Renshaw E.MCTest: A challenge dataset for the open-domain machine comprehension of text[J].Empirical Methods in Natural Language Processing,2013:193-203.
[3] Lai G, Xie Q, Liu H, et al. Race: Large-scale reading comprehension dataset from examinations[J].arXiv preprint arXiv: 1704.04683,2017.
[4] Rajpurkar P,et al.SQuAD: 100,000+questions for machine comprehension of text[J].Empirical Methods in Natural Language Processing,2016:2383-2392.
[5] 关勇,吕国英,李茹,等. 面向高考语文阅读理解的篇章标题选择研究[J].中文信息学报, 2018, 32(6): 28-35,43.
[6] Zhang S, Zhao H, Wu Y, et al. Dual co-matching network for multi-choice reading comprehension[J]. arXiv preprint arXiv: 1901.09381, 2019.
[7] Tan C, Wei F, Wang W, et al. Multiway attention networks for modeling sentence pairs[C]//Proceedings of IJCAI 2018. 2018:4411-4417.
[8] Seo M.Bi-directional attention flow for machine comprehension[C]//Proceedings of ICLR2017,2017:1-12.
[9] Hermann K M, Tomá Kocˇisky＇, Grefenstette E, et al. Teaching machines to read and comprehend[C]//Proceedings of the 28th International conference on Neural Information Processing Systems, 2015:1693-1701.
[10] 张浩宇,张鹏飞,李真真,等. 基于自注意力机制的阅读理解模型[J]. 中文信息学报, 2018, 32(12): 125-131.
[11] Hochreiter S, Schmidhuber J. Long short-term memory[J].Neural Computation,1997,9(8): 1735-1780.
[12] Cho K,Van Merrinboer B,Gulcehre C,et al. Learning phrase representations using RNN encoder-decoder for statistical machine translation[J]. Empirical Methods in Natural Language Processing,2014:1724-1734.
[13] Kadlec R, Schmid M, Bajgar O, et al. Text understanding with the attention sum reader network[J]. arXiv preprint arXiv: 1603.01547, 2016.
[14] Chen D.A thorough examination of the CNN/daily mail reading comprehension task[J].arXiv preprint arXiv: 1606.02838,2016.
[15] Dhingra B, Liu H, Yang Z, et al. Gated-attention readers for text comprehension[J].arXiv preprint arXiv: 1606.01549, 2016.
[16] Tseng B H, Shen S S, Lee H Y, et al.Towards machine comprehension of spoken content: Initial TOEFL listening comprehension test by machine[J]. arXiv preprint arXiv: 1608.06378, 2016.
[17] Cui Y, Liu T, Chen Z, et al.Consensus attention-based neural networks for Chinese reading comprehension[J].arXiv preprint arXiv: 1607.02250, 2016.
[18] Cui Y,et al.Attention-over-attention neural networks for reading comprehension[C]//Proceedings of ACL2017, 2017:593-602.
[19] Wang S, Jiang J. Machine comprehension using match-lstm and answer pointer[J].arXiv preprint arXiv: 1608.07905, 2016.
[20] Mikolov T, Sutskever I, Chen K, et al. Distributed representations of words and phrases and their compositionality[C]//Proceedings of the 26th Internations Neural Information Processing Systems, 2013:3111-3119.
[21] Kingma D P, Ba J. Adam: A method for stochastic optimization[J]. arXiv preprint arXiv: 1412.6980, 2014.
{{custom_fnGroup.title_cn}}
脚注
{{custom_fn.content}}
基金
山西省科技厅省基础研究计划项目(201801D121137)
{{custom_fund}}
PDF(4708 KB)
