基于深度交互融合网络的多跳机器阅读理解

朱斯琪,过弋,王业相

PDF(2249 KB)
PDF(2249 KB)
中文信息学报 ›› 2022, Vol. 36 ›› Issue (5) : 67-75.
机器阅读理解

基于深度交互融合网络的多跳机器阅读理解

  • 朱斯琪1,过弋1,2,3,王业相1
作者信息 +

Deep Interactive Fusion Network for Multi-hop Reading Comprehension

  • ZHU Siqi1, GUO Yi1,2,3, WANG Yexiang1
Author information +
History +

摘要

近年来,多跳机器阅读理解已经吸引了众多学者的关注,其要从多个文档中提取与问题相关的线索并回答问题。但很少有工作注重在段落选择时和回答问题时的多个段落之间的交互与融合,然而这对于多跳推理任务来说是至关重要的。因此,该文提出了一种针对多跳推理机器阅读理解的多段落深度交互融合的方法,首先从多个段落中筛选出与问题相关的段落,然后将得到的“黄金段落”输入到一个深度交互融合的网络中以聚集不同段落之间的信息,最终得到问题的答案。该文实验基于HotpotQA数据集,所提方法与基准模型相比,精确匹配(EM)提升18.5%,F1值提升18.47%。

Abstract

Multi-hop reading comprehension requiring information from multiple documents has attracted much attention. However, the interaction between paragraphs is less addressed, no matter in the gold paragraph selection or in question answering. In this paper, we propose a multi-paragraph deep interactive fusion network for multi-hop reading comprehension. First, we filter out paragraphs irrelevant to the query to reduce the impact of distractors on model performance. Then, the selected documents are further input to a deep interactive fusion network to aggregate information from different paragraphs for the final answer. Experiment on HotpotQA dataset demonstrates that our model achieves the improvements of 18.5% according to EM and 18.47% according to F1-score compared with the baseline.

关键词

多跳推理 / 机器阅读理解 / 多段落融合

Key words

multi-hop reasoning / machine reading comprehension / multi-paragraph interactive fusion

引用本文

导出引用
朱斯琪,过弋,王业相. 基于深度交互融合网络的多跳机器阅读理解. 中文信息学报. 2022, 36(5): 67-75
ZHU Siqi, GUO Yi, WANG Yexiang. Deep Interactive Fusion Network for Multi-hop Reading Comprehension. Journal of Chinese Information Processing. 2022, 36(5): 67-75

参考文献

[1] Pranav R, Jian Zh, Konstantin L, et al. SQuAD: 100,000+ questions for machine comprehension of text[C]//Proceedings of the Conference on Empirical Methods in Natural Language Processing. Austin, Texas: Association for Computational Linguistics,2016: 2383-2392.
[2] Pranav R, Robin J, Percy L. Know what you don’t know: unanswerable questions for SQuAD[C]//Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics. Melbourne, Australia: Association for Computational Linguistics,2018: 784-789.
[3] Guokun L, Qizhe X, Hanxiao L, et al. RACE: large-scale reading comprehension dataset from examinations[C]//Proceedings of the Conference on Empirical Methods in Natural Language Processing. Copenhagen, Denmark: Association for Computational Linguistics,2017: 785-794.
[4] Siva R,Danqi Ch, Christopher D M. CoQA: a conversational question answering challenge[C]//Proceedings of the Transactions of the Association for Computational Linguistics,2019: 249-266.
[5] He W, Liu K, Liu J, et al.DuReader: a Chinese machine reading comprehension dataset from real-world applications[J]. arXiv preprint arXiv: 1711.05073, 2017.
[6] Johannes W, Pontus S, Sebastian R. Constructing datasets for multi-hop reading comprehension across documents[C]//Proceedings of the Transactions of the Association for Computational Linguistics,2018: 287-302.
[7] Alon T, Jonathan B. The web as a knowledge-base for answering complex questions[C]//Proceedings of the Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. New Orleans, Louisiana,2018: 641-651.
[8] Zhilin Y, Peng Q,Saizheng Zh, et al. HotpotQA: a dataset for diverse, explainable multi-hop question answering[C]//Proceedings of the Conference on Empirical Methods in Natural Language Processing. Brussels, Belgium: Association for Computational Linguistics,2018: 2369-2380.
[9] Fang Y, Sun S, Gan Z, et al. Hierarchical graph network for multi-hop question answering[J].arXiv preprint arXiv: 1911.03631, 2019.
[10] Devlin J, Chang M W, Lee K, et al. Bert: Pretraining of deep bidirectional transformers for language understanding[J].arXiv preprint arXiv:1810.04805, 2018.
[11] Liu Y, Ott M, Goyal N, et al. Roberta: a robustly optimized bert pretraining approach[J]. arXiv preprint arXiv:1907.11692, 2019.
[12] Lan Z, Chen M, Goodman S, et al. Albert: a litebert for self-supervised learning of language representations[J]. arXiv preprint arXiv:1909.11942, 2019.
[13] Sewon M, Victor Zh, Luke Z, et al. Multi-hop reading comprehension through question decomposition and rescoring[C]//Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics. Florence, Italy: Association for Computational Linguistics,2019: 6097-6109.
[14] Perez E, Lewis P,Yih W, et al. Unsupervised question decomposition for question answering[J]. arXiv preprint arXiv:2002.09758, 2020.
[15] Nishida K, Nishida K, Nagata M, et al. Answering while summarizing: multi-task learning for multi-hop QA with evidence extraction[J].arXiv preprint arXiv:1905.08511, 2019.
[16] Tu M, Huang K, Wang G, et al. Select, answer and explain: interpretable multi-hop reading comprehension over multiple documents[C]//Proceedings of the AAAI, 2020: 9073-9080.
[17] Groeneveld D,Khot T, Sabharwal A. A simple yet strong pipeline for HotpotQA[J]. arXiv preprint arXiv:2004.06753, 2020.
[18] Sewon M, Eric W, Sameer S, et al. Compositional questions do not necessitate multi-hop reasoning[C]//Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics. Florence, Italy: Association for Computational Linguistics,2019: 4249-4257.
[19] Shao N, Cui Y, Liu T, et al. Is graph structure necessary for multi-hop reasoning?[J]. arXiv preprint arXiv:2004.03096, 2020.
[20] Jiang Y, Bansal M. Self-assembling modular networks for interpretable multi-hop reasoning[J].arXiv preprint arXiv:1909.05803, 2019.
[21] Dhingra B,Jin Q, Yang Z, et al. Neural models for reasoning over multiple mentions using coreference[J]. arXiv preprint arXiv:1804.05922, 2018.
[22] Ming T,Guangtao W, Jing H, et al. Multi-hop reading comprehension across multiple documents by reasoning over heterogeneous graphs [C]//Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics. Florence, Italy: Association for Computational Linguistics,2019:2704-2713.
[23] Ming D, Chang Zh,Qibin Ch, et al. Cognitive graph for multi-hop reading comprehension at scale[C]//Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics. Florence, Italy: Association for Computational Linguistics,2019:2694-2703.
[24] Nicola D C,Wilker A, Ivan T. Question answering by reasoning across documents with graph convolutional networks[C]//Proceedings of the Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. Minneapolis, Minnesota: Association for Computational Linguistics, 2019: 2306-2317.
[25] Christopher D M, Mihai S, John B, et al. The stanford corenlp natural language processing toolkit[C]//Proceedings of 52nd Annual Meeting of the Association for Computational Linguistics: System Demonstrations.2014: 55-60.
[26] Kipf T N, W M. Semi-supervised classification with graph convolutional networks[J]. arXiv preprint arXiv:1609.02907, 2016.
[27] Velic^k P, Cucurull G, Casanova A, et al. Graph attention networks[J]. arXiv preprint arXiv:1710.10903, 2017.
[28] Lin Q, Yunxuan X, Yanru Q, et al. Dynamically fused graph network for multi-hop reasoning[C]//Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics. Florence, Italy: Association for Computational Linguistics, 2019: 6140-6150.
[29] Ashish V, Noam S, Niki P, et al. Attention is all you need[C]//Proceedings of advances in neural information processing systems.2017: 5998-6008.
[30] Kaiming H, Xiangyu Zh, Shaoqing R, et al. Deep residual learning for image recognition[C]//Proceedings of the IEEE, Conference on Computer Vision and Pattern Recognition. 2016: 770-778.
[31] Seo M, Kembhavi A, Farhadi A, et al. Bidirectional attention flow for machine comprehension[J]. arXiv preprint arXiv:1611.01603, 2016.
[32] Xiong C, Zhong V, Socher R. Dynamic coattention networks for question answering[J]. arXiv preprint arXiv:1611.01604, 2016.
[33] Kingma D P, Ba J. Adam: a method for stochastic optimization[J]. arXiv preprint arXiv:1412.6980, 2014.
[34] Ye D, Lin Y, Liu Z, et al. Multi-paragraph reasoning with knowledge-enhanced graph neural network[J]. arXiv preprint arXiv:1911.02170, 2019.
[35] Chen J, Lin S, Durrett G. Multi-hop question answering via reasoning chains[J].arXiv preprint arXiv:1910.02610, 2019.
[36] Cui Y, Chen Z, Wei S, et al. Attention-over-attention neural networks for reading comprehension[J].arXiv preprint arXiv:1607.04423, 2016.
[37] Dua D, Wang Y, Dasigi P, et al. DROP: A reading comprehension benchmark requiring discrete reasoning over paragraphs[J]. arXiv preprint arXiv:1903.00161, 2019.

基金

国家重点研发计划(2018YFC0807105);国家自然科学基金(61462073);上海市科学技术委员会科研计划项目(17DZ1101003,18511106602, 18DZ2252300)
PDF(2249 KB)

1183

Accesses

0

Citation

Detail

段落导航
相关文章

/