倪艺函,兰艳艳,庞亮,程学旗. 多跳式文本阅读理解方法综述[J]. 中文信息学报, 2022, 36(11): 1-19.
NI Yihan, LAN Yanyan, PANG Liang, CHENG Xueqi. A Survey of Multi-hop Reading Comprehension for Text. , 2022, 36(11): 1-19.
A Survey of Multi-hop Reading Comprehension for Text
NI Yihan1,2, LAN Yanyan3, PANG Liang1,2, CHENG Xueqi1,2
1.CAS Key Lab of Network Data Science and Technology, Institute of Computing Technology, Chinese Academy of Sciences, Beijing 100190, China; 2.School of Computer and Control Engineering, University of Chinese, Beijing 100049, China; 3.Institute for AI Industry Research, Tsinghua University, Beijing 100084, China
Abstract：In recent years, multi-hop reading comprehension has become a hot topic in natural language understanding. Compared with single-hop reading comprehension, multi-hop reading comprehension is more challenging considering the involvement of multiple clues (e.g. multiple documents reading) the expectation for explicit reasoning paths. This paper summarizes the multi-hop reading comprehension task. This paper first gives the definition of multi-hop reading comprehension task. And then, and according to different reasoning methods, multi-hop reading comprehension models are can be divided into three categories: models based on structured reasoning, models based on evidence extraction, and models based on question decomposition. This paper analyzes the experimental results of these models on the common multi-hop reading comprehension datasets, revealing their advantages and disadvantages. Finally, the future research directions are discussed.
[1] YANG Z, QI P, ZHANG S, et al. HotpotQA: A dataset for diverse, explainable multi-hop question answering[C]//Proceedings of the Conference on Empirical Methods in Natural Language Processing, 2018: 2369-2380. [2] ZHAO C, XIONG C, ROSSET C, et al. Transformer-Xh: Multi-evidence reasoning with extra hop attention[C]//Proceedings of the International Conference on Learning Representations, 2019. [3] KIM D, KIM S,KWAK N. Textbook question answering with multi-modal context graph understanding and self-supervised open-set comprehension[C]//Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, 2019: 3568-3584. [4] TU M, HUANG K, WANG G et al. Select, answer and explain: Interpretable multi-hop reading comprehension over multiple documents[C]//Proceedings of the 34th AAAI Conference on Artificial Intelligence, 2020: 9073-9080. [5] DE CAO N, AZIZ W,TITOV I. Question answering by reasoning across documents with graph convolutional networks[C]//Proceedings of the Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Minneapolis, MN, USA: Association for Computational, 2019: 2306-2317. [6] SUN H, BEDRAXWEISS T,COHEN W. PullNET: Open domain question answering with iterative retrieval on knowledge bases and text[C]//Proceedings of the Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing, 2019: 2380-2390. [7] YE D, LIN Y, LIU Z, et al. Multi-paragraph reasoning with knowledge-enhanced graph neural network[J/OL]. arXiv preprint arXiv: 1911.02170, 2019. [8] LONG A, MASON J, BLAIR A, et al. Multi-hop reading comprehension via deep reinforcement learning based document traversal[J/OL]. arXiv preprint arXiv: 1905.09438, 2019. [9] TU M, WANG G, HUANG J, et al. Multi-hop reading comprehension across multiple documents by reasoning over heterogeneous graphs[C]//Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, 2019: 2704-2713. [10] TANG Z, SHEN Y, MA X, et al. Multi-hop reading comprehension across documents with path-based graph convolutional network[J/OL]. arXiv preprint arXiv: 2006.06478, 2020. [11] THAYAPARAN M, VALENTINO M, SCHLEGEL V, et al. Identifying supporting facts for multi-hop question answering with document graph networks[C]//Proceedings of the 13th Workshop on Graph-Based Methods for Natural Language Processing, 2019: 42-51. [12] FANG Y, SUN S, GAN Z, et al. Hierarchical graph network for multi-hop question answering[C]//Proceedings of the Conference on Empirical Methods in Natural Language Processing. Online: Association for Computational Linguistics, 2020: 8823-8838. [13] TU M, HUANG J, HE X, et al. Graph sequential network for reasoning over sequences[J/OL]. arXiv preprint arXiv: 2004.02001, 2020. [14] SONG L, WANG Z, YU M, et al. Exploring graph-structured passage representation for multi-hop reading comprehension with graph neural networks[J/OL]. arXiv preprint arXiv: 1809.02040, 2018. [15] QIU L, XIAO Y, QU Y, et al. Dynamically fused graph network for multi-hop reasoning[C]//Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, 2019: 6140-6150. [16] ZHANG Y, NIE P, RAMAMURTHY A, et al. DDRQA: Dynamic document reranking for open-domain multi-hop question answering[J/OL]. arXiv preprint arXiv: 2009.07465, 2020. [17] DING M, ZHOU C, CHEN Q, et al. Cognitive graph for multi-hop reading comprehension at scale[C]//Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, 2019: 2694-2703. [18] CAO Y, FANG M,TAO D. Bag: Bidirectional attention entity graph convolutional network for multi-hop reasoning question answering[C]//Proceedings of the Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Minneapolis, MN, USA: Association for Computational, 2019: 357-362. [19] MA X, ZHU Q, ZHOU Y, et al. Asking complex questions with multi-hop answer-focused reasoning[J/OL]. arXiv preprint arXiv: 2009.07402, 2020. [20] 舒冲, 欧阳智, 杜逆索, 等. 基于改进图节点的图神经多跳阅读理解研究[J/OL]. 计算机工程, 2020: 1-9 10.19678/j.issn.1000-3428.0059917[2021-05-07]. [21] KUNDU S, KHOT T, SABHARWAL A, et al. Exploiting explicit paths for multi-hop reading comprehension[C]//Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, 2019: 2737-2747. [22] JIANG Y, JOSHI N, CHEN Y C, et al. Explore, propose and assemble: An interpretable model for multi-hop reading comprehension[C]//Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, 2019: 2714-2725. [23] FENG Y, YU M, XIONG W, et al. Learning to recover reasoning chains for multi-hop question answering via cooperative games[J/OL]. arXiv preprint arXiv: 2004.02393, 2020. [24] ASAI A, HASHIMOTO K, HAJISHIRZI H, et al. Learning to retrieve reasoning paths over wikipedia graph for question answering[C]//Proceedings of the International Conference on Learning Representations, 2019. [25] CHEN J, LIN S T, DURRETT G. Multi-hop question answering via reasoning chains[J/OL]. arXiv preprint arXiv: 1910.02610, 2019. [26] QI P, LIN X, MEHR L, et al. Answering complex open-domain questions through iterative query generation[C]//Proceedings of the International Joint Nonference on Natural Language Processing, 2019: 2590-2602. [27] NISHIDA K, NISHIDA K, NAGATA M, et al. Answering while summarizing: Multi-task learning for multi-hop qa with evidence extraction[C]//Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, 2019: 2335-2345. [28] DHINGRA B, ZAHEER M, BALACHANDRAN V, et al. Differentiable reasoning over a virtual knowledge base[C]//Proceedings of the International Conference on Learning Representations, 2019. [29] FELDMAN Y,ELYANIV R. Multi-hop paragraph retrieval for open-domain question answering[C]//Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, 2019: 2296-2309. [30] DAS R, GODBOLE A, KAVARTHAPU D, et al. Multi-step entity-centric information retrieval for multi-hop question answering[C]//Proceedings of the 2nd Workshop on Machine Reading for Question Answering, 2019: 113-118. [31] DAS R, DHULIAWALA S, ZAHEER M, et al. Multi-step retriever-reader interaction for scalable open-domain question answering[C]//Proceedings of the International Conference on Learning Representations, 2018. [32] XIONG W, YU M, GUO X, et al. Simple yet effective bridge reasoning for open-domain multi-hop question answering[C]//Proceedings of the 2nd Workshop on Machine Reading for Question Answering, 2019: 48-52. [33] YADAV V, BETHARD S,SURDEANU M. Unsupervised alignment-based iterative evidence retrieval for multi-hop question answering[C]//Proceedings of the 58th Annual Meeting of the Association for Computational LinguisticsAssociation for Computational Linguistics, 2020: 4514-4525. [34] GROENEVELD D, KHOT T, SABHARWAL A, et al. A simple yet strong pipeline for hotpotqa[C]//Proceedings of the Conference on Empirical Methods in Natural Language Processing. Online: Association for Computational Linguistics, 2020: 8839-8845. [35] NIE Y, WANG S,BANSAL M. Revealing the importance of semantic retrieval for machine reading at scale[C]//Proceedings of the Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing, 2019: 2553-2566. [36] TRIVEDI H, KWON H, KHOT T, et al. Repurposing entailment for multi-hop question answering tasks[C]//Proceedings of the Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. Minneapolis, MN, USA: Association for Computational, 2019: 2948-2958. [37] YADAV V, BETHARD S,SURDEANU M. Quick and (not so) dirty: Unsupervised selection of justification sentences for multi-hop question answering[C]//Proceedings of the Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing. Hong Kong, China: Association for Computational Linguistics, 2019: 2578-2589. [38] 段艺文. 多层注意机制下阅读理解问答模型研究与应用[D]. 成都: 电子科技大学硕士学位论文, 2020. [39] 李天仙. 基于多跳注意力的中文机器阅读理解[D]. 武汉: 华中师范大学硕士学位论文, 2020. [40] 盛艺暄,兰曼. 利用外部知识辅助和多步推理的选择题型机器阅读理解模型[J]. 计算机系统应用, 2020, 29(4): 1-9. [41] KUMAR A, IRSOY O, ONDRUSKA P, et al. Ask me anything: Dynamic memory networks for natural language processing[C]//Proceedings of the International Conference on Machine Learning, 2016: 1378-1387. [42] ZHONG V, XIONG C, KESKAR N S, et al. Coarse-grain fine-grain coattention network for multi-evidence question answering[C]//Proceedings of the International Conference on Learning Representations, 2019. [43] BAUER L, WANG Y,BANSAL M. Commonsense for generative multi-hop question answering tasks[C]//Proceedings of the Conference on Empirical Methods in Natural Language Processing, 2018: 4220-4230. [44] SHEN Y, LIU X, DUH K, et al. An empirical analysis of multiple-turn reasoning strategies in reading comprehension tasks[C]//Proceedings of the 8th International Joint Conference on Natural Language Processing, 2017: 957-966. [45] SUKHBAATAR S, WESTON J,FERGUS R. End-to-end memory networks[C]//Proceedings of the 29th International Conference on Neural Information Processing Systems, 2015: 2440-2448. [46] AINSLIE J, ONTANON S, ALBERTI C, et al. ETC: Encoding long and structured inputs in transformers[C]//Proceedings of the Conference on Empirical Methods in Natural Language Processing, 2020: 268-284. [47] BELTAGY I, PETERS M E,COHAN A. Longformer: the long-document transformer[J/OL]. arXiv preprint arXiv: 2004.05150, 2020. [48] WESTON J, CHOPRA S,BORDES A. Memory networks[C]//Proceedings of the International Conference on Learning Representations, 2015. [49] YU S, INDURTHI S R, BACK S, et al. A multi-stage memory augmented neural network for machine reading comprehension[C]//Proceedings of the Workshop on Machine Reading for Question Answering, 2018: 21-30. [50] SHEN Y, HUANG P S, GAO J, et al. ReasoNet: Learning to stop reading in machine comprehension[C]//Proceedings of the 23rd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, 2017: 1047-1055. [51] GONG Y,BOWMAN S. Ruminating Reader: Reasoning with gated multi-hop attention[C]//Proceedings of the Workshop on Machine Reading for Question Answering, 2018: 1-11. [52] PEREZ E, LEWIS P, YIH W-T, et al. Unsupervised question decomposition for question answering[C]//Proceedings of the Conference on Empirical Methods in Natural Language Processing. Online: Association for Computational Linguistics,2020: 8864-8880. [53] KHOT T, KHASHABI D, RICHARDSON K, et al. Text modular networks: Learning to decompose tasks in the language of existing models[J/OL]. arXiv preprint arXiv: 2009.00751, 2020. [54] MIN S, ZHONG V, ZETTLEMOYER L, et al. Multi-hop reading comprehension through question decomposition and rescoring[C]//Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, 2019: 6097-6109. [55] TRAN N K,NIEDERE C. Multihop attention networks for question answer matching[C]//Proceedings of the 41st International ACM SIGIR Conference on Research and Development in Information Retrieval, 2018: 325-334. [56] JIANG Y,BANSAL M. Self-assembling modular networks for interpretable multi-hop reasoning[C]//Proceedings of the Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing, 2019: 4464-4474. [57] GRAIL Q, PEREZ J,GAUSSIER E. Latent question reformulation and information accumulation for multi-hop machine reading[DB/OL]. https://openreview.net/pdf?id=S1x63TEYvr[2021-05-16]. [58] GUPTA N, LIN K, ROTH D, et al. Neural module networks for reasoning over text[C]//Proceedings of the International Conference on Learning Representations, 2019. [59] WEBER L, MINERVINI P, MNCHMEYER J, et al. NlProlog: Reasoning with weak unification for question answering in natural language[C]//Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, 2019: 6151-6161. [60] JIANG Y,BANSAL M. Avoiding reasoning shortcuts: Adversarial evaluation, training, and model development for multi-hop[C]//Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, 2019: 2726-2736. [61] MIN S, WALLACE E, SINGH S, et al. Compositional questions do not necessitate multi-hop reasoning[C]//Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics,2019: 4249-4257. [62] TRIVEDI H, BALASUBRAMANIAN N, KHOT T, et al. Measuring and reducing non-multifact reasoning in multi-hop question answering[J/OL]. arXiv preprint arXiv: 2005.00789, 2020. [63] TANG Y, NG H T,TUNG A. Do multi-hop question answering systems know how to answer the single-hop sub-questions?[C]//Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics: Main Volume. Online: Association for Computational Linguistics, 2021: 3244-3249. [64] WANG H, YU M, GUO X, et al. Do multi-hop readers dream of reasoning chains?[C]//Proceedings of the 2nd Workshop on Machine Reading for Question Answering, 2019: 91-97. [65] CHEN J,DURRETT G. How to learn (and how not to learn) multi-hop reasoning with memory networks[DB/OL]. https://openreview.net/pdf?id=B1lf43A5Y7[2021-05-07]. [66] SHAO N, CUI Y, LIU T, et al. Is graph structure necessary for multi-hop question answering?[C]//Proceedings of the Conference on Empirical Methods in Natural Language Processing. Online: Association for Computational Linguistics, 2020: 7187-7192. [67] USTER S, SUSHIL M,DAELEMANS W. Why cant memory networks read effectively?[J/OL]. arXiv preprint arXiv: 1910.07350, 2019. [68] WELBL J, STENETORP P,RIEDEL S. Constructing datasets for multi-hop reading comprehension across documents[J]. Transactions of the Association for Computational Linguistics, 2018, 6: 287-302. [69] CHEN J,DURRETT G. Understanding dataset design choices for multi-hop reasoning[C]//Proceedings of the North American Chapter of the Association for Computational Linguistics, 2019: 4026-4032. [70] WESTON J, BORDES A, CHOPRA S, et al. Towards ai-complete question answering: a set of prerequisite toy tasks[C]//Proceedings of the International Conference on Learning Representations, 2016. [71] CHEN W, ZHA H, CHEN Z, et al. HybridQA: A dataset of multi-hop question answering over tabular and textual data[C]//Proceedings of the Findings of the Association for Computational Linguistics: EMNLP, 2020: 1026-1036. [72] TALMOR A,BERANT J. The web as a knowledge-base for answering complex questions[C]//Proceedings of the Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, New Orleans, Louisiana, USA: Association for Computational Linguistics, 2018: 641-651. [73] ABUJABAL A, ROY R S, YAHYA M, et al. ComQA: A community-sourced dataset for complex factoid question answering with paraphrase clusters[C]//Proceedings of the Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. Minneapolis, MN, USA: Association for Computational, 2019: 307-317. [74] KHOT T, CLARK P, GUERQUIN M, et al. QASC: A dataset for question answering via sentence composition[C]//Proceedings of the 34th AAAI Conference on Artificial Intelligence, 2020. [75] WOLFSON T, GEVA M, GUPTA A, et al. Break it down: A question understanding benchmark[J]. Transactions of the Association for Computational Linguistics, 2020, 8: 183-198. [76] KOCˇISKY＇ T, SCHWARZ J, BLUNSOM P, et al. The narrativeqa reading comprehension challenge[J]. Transactions of the Association for Computational Linguistics, 2018, 6: 317-328. [77] KHASHABI D, CHATURVEDI S, ROTH M, et al. Looking beyond the surface: A challenge set for reading comprehension over multiple sentences[C]//Proceedings of the Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. New Orleans, Louisiana, USA: Association for Computational Linguistics, 2018: 252-262. [78] DUA D, WANG Y, DASIGI P, et al. DROP: A reading comprehension benchmark requiring discrete reasoning over paragraphs[C]//Proceedings of the Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Minneapolis, MN, USA: Association for Computational, 2019: 2368-2378. [79] REDDY S, CHEN D,MANNING C D. COQA: A conversational question answering challenge[J]. Transactions of the Association for Computational Linguistics, 2019, 7: 249-266. [80] CHOI E, HE H, IYYER M, et al. QUAC: Question answering in context[C]//Proceedings of the Conference on Empirical Methods in Natural Language Processing, 2018: 2174-2184. [81] ELGOHARY A, ZHAO C,BOYD-GRABER J. a dataset and baselines for sequential open-domain question answering[C]//Proceedings of the Conference on Empirical Methods in Natural Language Processing, 2018: 1077-1083. [82] MIHAYLOV T, CLARK P, KHOT T, et al. Can a suit of armor conduct electricity?: A new dataset for open book question answering[C]//Proceedings of the Conference on Empirical Methods in Natural Language Processing, 2018: 2381-2391. [83] SAHA A, ARALIKATTE R, KHAPRA M M, et al. DuoRC: Towards complex language understanding with paraphrased reading comprehension[C]//Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics. Melbourne, Australia: Association for Computational Linguistics, 2018: 1683-1693. [84] SEO M, KEMBHAVI A, FARHADI A, et al. Bidirectional attention flow for machine comprehension[C]//Proceedings of the International Conference on Learning Representations, 2017. [85] DEVLIN J, CHANG M W, LEE K, et al. BERT: Pretraining of deep bidirectional transformers for language understanding[C]//Proceedings of the Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. Minneapolis, MN, USA: Association for Computational, 2019: 4171-4186. [86] TALMOR A,BERANT J. MultiQA: An empirical investigation of generalization and transfer in reading comprehension[C]//Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, 2019: 4911-4921. [87] WEISSENBORN D, WIESE G,SEIFFE L. Making neural QA as simple as possible but not simpler[C]//Proceedings of the 21st Conference on Computational Natural Language Learning, 2017: 271-280. [88] DHINGRA B, JIN Q, YANG Z, et al. Neural models for reasoning over multiple mentions using coreference[C]//Proceedings of the Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. New Orleans, Louisiana, USA: Association for Computational Linguistics, 2018: 42-48. [89] RADFORD A, WU J, CHILD R, et al. Language models are unsupervised multitask learners[DB/OL]. https://cdn.openai.com/better-language-models/language_models_are_unsupervised_multitask_learners.pdf[2021-05-07]. [90] YANG H, WANG H, GUO S, et al. Learning to decompose compound questions with reinforcement learning[DB/OL]. https://openreview.net/pdf?id=SJl2ps0qKQ [2021-05-07].
2022, Vol. 36 
