基于自适应知识选择的机器阅读理解

李泽政,田志兴,张元哲,刘康,赵军

PDF(4267 KB)
PDF(4267 KB)
中文信息学报 ›› 2022, Vol. 36 ›› Issue (6) : 117-124.
机器阅读理解

基于自适应知识选择的机器阅读理解

  • 李泽政1,2,田志兴1,2,张元哲1,刘康1,2,赵军1,2
作者信息 +

Adaptive Knowledge Selection for Machine Reading Comprehension

  • LI Zezheng1,2, TIAN Zhixing1,2, ZHANG Yuanzhe1, LIU Kang1,2, ZHAO Jun1,2
Author information +
History +

摘要

目前针对知识增强机器阅读理解的研究主要集中在如何把外部知识融入现有的机器阅读理解模型,却忽略了对外部知识的来源进行选择。该文首先基于注意力机制对外部知识进行编码,然后对不同来源的外部知识编码进行打分,最后自适应地选择出对回答问题最有帮助的知识。与基线模型相比,该文提出的基于自适应知识选择的机器阅读理解模型在准确率上提高了1.2个百分点。

Abstract

The current knowledge-enhanced machine reading comprehension is focused on how to integrate external knowledge into the existing MRC model, while ignores the selection for the source of external knowledge. This article first uses the attention mechanism to encode external knowledge, then scores external knowledge from different sources, and finally selects the most helpful knowledge with respect to different questions. Compared with the baseline models, our method improves the accuracy by 1.2 percent.

关键词

机器阅读理解 / 知识增强 / 自适应选择

Key words

machine reading comprehension / knowledge enhancement / adaptive selection

引用本文

导出引用
李泽政,田志兴,张元哲,刘康,赵军. 基于自适应知识选择的机器阅读理解. 中文信息学报. 2022, 36(6): 117-124
LI Zezheng, TIAN Zhixing, ZHANG Yuanzhe, LIU Kang, ZHAO Jun. Adaptive Knowledge Selection for Machine Reading Comprehension. Journal of Chinese Information Processing. 2022, 36(6): 117-124

参考文献

[1] Chen Q, Zhu X, Ling Z H, et al. Neural natural language inference models enhanced with external knowledge[J]. arXiv preprint arXiv:1711.04289, 2017.
[2] Wang L, Sun M, Zhao W, et al. Yuanfudao at semeval-2018 task 11: Three-way attention and relational knowledge for commonsense machine comprehension[J]. arXiv preprint arXiv:1803.00191, 2018.
[3] Yang A, Wang Q, Liu J, et al. Enhancing pre-trained language representations with rich knowledge for machine reading comprehension[C]//Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics. 2019: 2346-2357.
[4] Mihaylov T, Frank A. Knowledgeable reader: Enhancing cloze-style reading comprehension with external commonsense knowledge[J]. arXiv preprint arXiv:1805.07858, 2018.
[5] Sun Y, Guo D, Tang D, et al. Knowledge based machine reading comprehension[J]. arXiv preprint arXiv:1809.04267, 2018.
[6] Weissenborn D, Kocˇisky' T, Dyer C. Dynamic integration of background knowledge in neural NLU systems[J]. arXiv preprint arXiv:1706.02596, 2017.
[7] Bauer L, Wang Y, Bansal M. Commonsense for generative multi-hop question answering tasks[J]. arXiv preprint arXiv:1809.06309, 2018.
[8] Miller G A. WordNet: A lexical database for English[J]. Communications of the ACM, 1995, 38(11): 39-41.
[9] Speer R, Chin J, Havasi C. ConceptNet 5.5: An open multilingual graph of general knowledge[C]//Proceedings of the AAAI Conference on Artificial Intelligence, 2017.
[10] Roth M, Mostafazadeh N, Chambers N, et al. LSDSem 2017 shared task: The story cloze test[C]//Proceedings of the 2nd Workshop on Linking Models of Lexical, Sentential and Discourse-level Semantics, 2017.
[11] Richardson M, Burges C J C, Renshaw E. Mctest: A challenge dataset for the open-domain machine comprehension of text[C]//Proceedings of the Conference on Empirical Methods in Natural Language Processing, 2013: 193-203.
[12] Chen D, Bolton J, Manning C D. A thorough examination of the cnn/daily mail reading comprehension task[J]. arXiv preprint arXiv:1606.02858, 2016.
[13] Rajpurkar P, Zhang J, Lopyrev K, et al. Squad: 100,000+ questions for machine comprehension of text[J]. arXiv preprint arXiv:1606.05250, 2016.
[14] Wang S, Jiang J. Machine comprehension using match-lstm and answer pointer[J]. arXiv preprint arXiv:1608.07905, 2016.
[15] Seo M, Kembhavi A, Farhadi A, et al. Bidirectional attention flow for machine comprehension[J]. arXiv preprint arXiv:1611.01603, 2016.
[16] Wang W, Yang N, Wei F, et al. Gated self-matching networks for reading comprehension and question answering[C]//Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics, 2017: 189-198.
[17] Long T, Bengio E, Lowe R, et al. World knowledge for reading comprehension: rare entity prediction with hierarchical LSTMs using external descriptions[C]//Proceedings of the Conference on Empirical Methods in Natural Language Processing, 2017: 825-834.
[18] Mihaylov T, Frank A. Knowledgeable reader: Enhancing cloze-style reading comprehension with external commonsense knowledge[J]. arXiv preprint arXiv:1805.07858, 2018.
[19] Wang C, Jiang H. Exploring machine reading comprehension with explicit knowledge[J]. arXiv preprint arXiv:1809.03449, 2018.
[20] Pan X, Sun K, Yu D, et al. Improving question answering with external knowledge[J]. arXiv preprint arXiv:1902.00993, 2019.
[21] Li Q, Li Z, Wei J M, et al. A multi-attention based neural network with external knowledge for story ending predicting task[C]//Proceedings of the 27th International Conference on Computational Linguistics, 2018: 1754-1762.
[22] Chen J, Chen J, Yu Z. Incorporating structured commonsense knowledge in story completion[C]//Proceedings of the AAAI Conference on Artificial Intelligence, 2019, 33(01): 6244-6251.
[23] Tian Z, Zhang Y, Liu K, et al. Scene restoring for narrative machine reading comprehension[C]//Proceedings of the Conference on Empirical Methods in Natural Language Processing, 2020: 3063-3073.
[24] Devlin J, Chang M W, Lee K, et al. Bert: Pre-training of deep bidirectional transformers for language understanding[J]. arXiv preprint arXiv:1810.04805, 2018.
[25] Cai Z, Tu L, Gimpel K. Pay attention to the ending: Strong neural baselines for the roc story cloze task[C]//Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics, 2017: 616-622.
[26] Chaturvedi S, Peng H, Roth D. Story comprehension for predicting what happens next[C]//Proceedings of the Conference on Empirical Methods in Natural Language Processing, 2017: 1603-1614.
[27] Cui Y, Che W, Zhang W N, et al. Discriminative sentence modeling for story ending prediction[C]//Proceedings of the AAAI Conference on Artificial Intelligence, 2020, 34(05): 7602-7609.
[28] Kingma D P, Ba J. Adam: A method for stochastic optimization[J]. arXiv preprint arXiv:1412.6980, 2014.
[29] Radford A, Narasimhan K, Salimans T, et al. Improving language understanding by generative pre-training[R]. Open AI, 2018.

基金

国家重点研发计划项目(2018YFB1005100)
PDF(4267 KB)

1332

Accesses

0

Citation

Detail

段落导航
相关文章

/