|
|
|
| A Survey of Multi-hop Reading Comprehension for Text |
| NI Yihan1,2, LAN Yanyan3, PANG Liang1,2, CHENG Xueqi1,2 |
1.CAS Key Lab of Network Data Science and Technology, Institute of Computing Technology, Chinese Academy of Sciences, Beijing 100190, China;
2.School of Computer and Control Engineering, University of Chinese, Beijing 100049, China;
3.Institute for AI Industry Research, Tsinghua University, Beijing 100084, China |
|
|
|
|
Abstract In recent years, multi-hop reading comprehension has become a hot topic in natural language understanding. Compared with single-hop reading comprehension, multi-hop reading comprehension is more challenging considering the involvement of multiple clues (e.g. multiple documents reading) the expectation for explicit reasoning paths. This paper summarizes the multi-hop reading comprehension task. This paper first gives the definition of multi-hop reading comprehension task. And then, and according to different reasoning methods, multi-hop reading comprehension models are can be divided into three categories: models based on structured reasoning, models based on evidence extraction, and models based on question decomposition. This paper analyzes the experimental results of these models on the common multi-hop reading comprehension datasets, revealing their advantages and disadvantages. Finally, the future research directions are discussed.
|
|
Received: 10 March 2021
|
|
|
|
|
|
[1] YANG Z, QI P, ZHANG S, et al. HotpotQA: A dataset for diverse, explainable multi-hop question answering[C]//Proceedings of the Conference on Empirical Methods in Natural Language Processing, 2018: 2369-2380.
[2] ZHAO C, XIONG C, ROSSET C, et al. Transformer-Xh: Multi-evidence reasoning with extra hop attention[C]//Proceedings of the International Conference on Learning Representations, 2019.
[3] KIM D, KIM S,KWAK N. Textbook question answering with multi-modal context graph understanding and self-supervised open-set comprehension[C]//Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, 2019: 3568-3584.
[4] TU M, HUANG K, WANG G et al. Select, answer and explain: Interpretable multi-hop reading comprehension over multiple documents[C]//Proceedings of the 34th AAAI Conference on Artificial Intelligence, 2020: 9073-9080.
[5] DE CAO N, AZIZ W,TITOV I. Question answering by reasoning across documents with graph convolutional networks[C]//Proceedings of the Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Minneapolis, MN, USA: Association for Computational, 2019: 2306-2317.
[6] SUN H, BEDRAXWEISS T,COHEN W. PullNET: Open domain question answering with iterative retrieval on knowledge bases and text[C]//Proceedings of the Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing, 2019: 2380-2390.
[7] YE D, LIN Y, LIU Z, et al. Multi-paragraph reasoning with knowledge-enhanced graph neural network[J/OL]. arXiv preprint arXiv: 1911.02170, 2019.
[8] LONG A, MASON J, BLAIR A, et al. Multi-hop reading comprehension via deep reinforcement learning based document traversal[J/OL]. arXiv preprint arXiv: 1905.09438, 2019.
[9] TU M, WANG G, HUANG J, et al. Multi-hop reading comprehension across multiple documents by reasoning over heterogeneous graphs[C]//Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, 2019: 2704-2713.
[10] TANG Z, SHEN Y, MA X, et al. Multi-hop reading comprehension across documents with path-based graph convolutional network[J/OL]. arXiv preprint arXiv: 2006.06478, 2020.
[11] THAYAPARAN M, VALENTINO M, SCHLEGEL V, et al. Identifying supporting facts for multi-hop question answering with document graph networks[C]//Proceedings of the 13th Workshop on Graph-Based Methods for Natural Language Processing, 2019: 42-51.
[12] FANG Y, SUN S, GAN Z, et al. Hierarchical graph network for multi-hop question answering[C]//Proceedings of the Conference on Empirical Methods in Natural Language Processing. Online: Association for Computational Linguistics, 2020: 8823-8838.
[13] TU M, HUANG J, HE X, et al. Graph sequential network for reasoning over sequences[J/OL]. arXiv preprint arXiv: 2004.02001, 2020.
[14] SONG L, WANG Z, YU M, et al. Exploring graph-structured passage representation for multi-hop reading comprehension with graph neural networks[J/OL]. arXiv preprint arXiv: 1809.02040, 2018.
[15] QIU L, XIAO Y, QU Y, et al. Dynamically fused graph network for multi-hop reasoning[C]//Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, 2019: 6140-6150.
[16] ZHANG Y, NIE P, RAMAMURTHY A, et al. DDRQA: Dynamic document reranking for open-domain multi-hop question answering[J/OL]. arXiv preprint arXiv: 2009.07465, 2020.
[17] DING M, ZHOU C, CHEN Q, et al. Cognitive graph for multi-hop reading comprehension at scale[C]//Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, 2019: 2694-2703.
[18] CAO Y, FANG M,TAO D. Bag: Bidirectional attention entity graph convolutional network for multi-hop reasoning question answering[C]//Proceedings of the Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Minneapolis, MN, USA: Association for Computational, 2019: 357-362.
[19] MA X, ZHU Q, ZHOU Y, et al. Asking complex questions with multi-hop answer-focused reasoning[J/OL]. arXiv preprint arXiv: 2009.07402, 2020.
[20] 舒冲, 欧阳智, 杜逆索, 等. 基于改进图节点的图神经多跳阅读理解研究[J/OL]. 计算机工程, 2020: 1-9 10.19678/j.issn.1000-3428.0059917[2021-05-07].
[21] KUNDU S, KHOT T, SABHARWAL A, et al. Exploiting explicit paths for multi-hop reading comprehension[C]//Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, 2019: 2737-2747.
[22] JIANG Y, JOSHI N, CHEN Y C, et al. Explore, propose and assemble: An interpretable model for multi-hop reading comprehension[C]//Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, 2019: 2714-2725.
[23] FENG Y, YU M, XIONG W, et al. Learning to recover reasoning chains for multi-hop question answering via cooperative games[J/OL]. arXiv preprint arXiv: 2004.02393, 2020.
[24] ASAI A, HASHIMOTO K, HAJISHIRZI H, et al. Learning to retrieve reasoning paths over wikipedia graph for question answering[C]//Proceedings of the International Conference on Learning Representations, 2019.
[25] CHEN J, LIN S T, DURRETT G. Multi-hop question answering via reasoning chains[J/OL]. arXiv preprint arXiv: 1910.02610, 2019.
[26] QI P, LIN X, MEHR L, et al. Answering complex open-domain questions through iterative query generation[C]//Proceedings of the International Joint Nonference on Natural Language Processing, 2019: 2590-2602.
[27] NISHIDA K, NISHIDA K, NAGATA M, et al. Answering while summarizing: Multi-task learning for multi-hop qa with evidence extraction[C]//Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, 2019: 2335-2345.
[28] DHINGRA B, ZAHEER M, BALACHANDRAN V, et al. Differentiable reasoning over a virtual knowledge base[C]//Proceedings of the International Conference on Learning Representations, 2019.
[29] FELDMAN Y,ELYANIV R. Multi-hop paragraph retrieval for open-domain question answering[C]//Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, 2019: 2296-2309.
[30] DAS R, GODBOLE A, KAVARTHAPU D, et al. Multi-step entity-centric information retrieval for multi-hop question answering[C]//Proceedings of the 2nd Workshop on Machine Reading for Question Answering, 2019: 113-118.
[31] DAS R, DHULIAWALA S, ZAHEER M, et al. Multi-step retriever-reader interaction for scalable open-domain question answering[C]//Proceedings of the International Conference on Learning Representations, 2018.
[32] XIONG W, YU M, GUO X, et al. Simple yet effective bridge reasoning for open-domain multi-hop question answering[C]//Proceedings of the 2nd Workshop on Machine Reading for Question Answering, 2019: 48-52.
[33] YADAV V, BETHARD S,SURDEANU M. Unsupervised alignment-based iterative evidence retrieval for multi-hop question answering[C]//Proceedings of the 58th Annual Meeting of the Association for Computational LinguisticsAssociation for Computational Linguistics, 2020: 4514-4525.
[34] GROENEVELD D, KHOT T, SABHARWAL A, et al. A simple yet strong pipeline for hotpotqa[C]//Proceedings of the Conference on Empirical Methods in Natural Language Processing. Online: Association for Computational Linguistics, 2020: 8839-8845.
[35] NIE Y, WANG S,BANSAL M. Revealing the importance of semantic retrieval for machine reading at scale[C]//Proceedings of the Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing, 2019: 2553-2566.
[36] TRIVEDI H, KWON H, KHOT T, et al. Repurposing entailment for multi-hop question answering tasks[C]//Proceedings of the Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. Minneapolis, MN, USA: Association for Computational, 2019: 2948-2958.
[37] YADAV V, BETHARD S,SURDEANU M. Quick and (not so) dirty: Unsupervised selection of justification sentences for multi-hop question answering[C]//Proceedings of the Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing. Hong Kong, China: Association for Computational Linguistics, 2019: 2578-2589.
[38] 段艺文. 多层注意机制下阅读理解问答模型研究与应用[D]. 成都: 电子科技大学硕士学位论文, 2020.
[39] 李天仙. 基于多跳注意力的中文机器阅读理解[D]. 武汉: 华中师范大学硕士学位论文, 2020.
[40] 盛艺暄,兰曼. 利用外部知识辅助和多步推理的选择题型机器阅读理解模型[J]. 计算机系统应用, 2020, 29(4): 1-9.
[41] KUMAR A, IRSOY O, ONDRUSKA P, et al. Ask me anything: Dynamic memory networks for natural language processing[C]//Proceedings of the International Conference on Machine Learning, 2016: 1378-1387.
[42] ZHONG V, XIONG C, KESKAR N S, et al. Coarse-grain fine-grain coattention network for multi-evidence question answering[C]//Proceedings of the International Conference on Learning Representations, 2019.
[43] BAUER L, WANG Y,BANSAL M. Commonsense for generative multi-hop question answering tasks[C]//Proceedings of the Conference on Empirical Methods in Natural Language Processing, 2018: 4220-4230.
[44] SHEN Y, LIU X, DUH K, et al. An empirical analysis of multiple-turn reasoning strategies in reading comprehension tasks[C]//Proceedings of the 8th International Joint Conference on Natural Language Processing, 2017: 957-966.
[45] SUKHBAATAR S, WESTON J,FERGUS R. End-to-end memory networks[C]//Proceedings of the 29th International Conference on Neural Information Processing Systems, 2015: 2440-2448.
[46] AINSLIE J, ONTANON S, ALBERTI C, et al. ETC: Encoding long and structured inputs in transformers[C]//Proceedings of the Conference on Empirical Methods in Natural Language Processing, 2020: 268-284.
[47] BELTAGY I, PETERS M E,COHAN A. Longformer: the long-document transformer[J/OL]. arXiv preprint arXiv: 2004.05150, 2020.
[48] WESTON J, CHOPRA S,BORDES A. Memory networks[C]//Proceedings of the International Conference on Learning Representations, 2015.
[49] YU S, INDURTHI S R, BACK S, et al. A multi-stage memory augmented neural network for machine reading comprehension[C]//Proceedings of the Workshop on Machine Reading for Question Answering, 2018: 21-30.
[50] SHEN Y, HUANG P S, GAO J, et al. ReasoNet: Learning to stop reading in machine comprehension[C]//Proceedings of the 23rd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, 2017: 1047-1055.
[51] GONG Y,BOWMAN S. Ruminating Reader: Reasoning with gated multi-hop attention[C]//Proceedings of the Workshop on Machine Reading for Question Answering, 2018: 1-11.
[52] PEREZ E, LEWIS P, YIH W-T, et al. Unsupervised question decomposition for question answering[C]//Proceedings of the Conference on Empirical Methods in Natural Language Processing. Online: Association for Computational Linguistics,2020: 8864-8880.
[53] KHOT T, KHASHABI D, RICHARDSON K, et al. Text modular networks: Learning to decompose tasks in the language of existing models[J/OL]. arXiv preprint arXiv: 2009.00751, 2020.
[54] MIN S, ZHONG V, ZETTLEMOYER L, et al. Multi-hop reading comprehension through question decomposition and rescoring[C]//Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, 2019: 6097-6109.
[55] TRAN N K,NIEDERE C. Multihop attention networks for question answer matching[C]//Proceedings of the 41st International ACM SIGIR Conference on Research and Development in Information Retrieval, 2018: 325-334.
[56] JIANG Y,BANSAL M. Self-assembling modular networks for interpretable multi-hop reasoning[C]//Proceedings of the Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing, 2019: 4464-4474.
[57] GRAIL Q, PEREZ J,GAUSSIER E. Latent question reformulation and information accumulation for multi-hop machine reading[DB/OL]. https://openreview.net/pdf?id=S1x63TEYvr[2021-05-16].
[58] GUPTA N, LIN K, ROTH D, et al. Neural module networks for reasoning over text[C]//Proceedings of the International Conference on Learning Representations, 2019.
[59] WEBER L, MINERVINI P, MNCHMEYER J, et al. NlProlog: Reasoning with weak unification for question answering in natural language[C]//Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, 2019: 6151-6161.
[60] JIANG Y,BANSAL M. Avoiding reasoning shortcuts: Adversarial evaluation, training, and model development for multi-hop[C]//Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, 2019: 2726-2736.
[61] MIN S, WALLACE E, SINGH S, et al. Compositional questions do not necessitate multi-hop reasoning[C]//Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics,2019: 4249-4257.
[62] TRIVEDI H, BALASUBRAMANIAN N, KHOT T, et al. Measuring and reducing non-multifact reasoning in multi-hop question answering[J/OL]. arXiv preprint arXiv: 2005.00789, 2020.
[63] TANG Y, NG H T,TUNG A. Do multi-hop question answering systems know how to answer the single-hop sub-questions?[C]//Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics: Main Volume. Online: Association for Computational Linguistics, 2021: 3244-3249.
[64] WANG H, YU M, GUO X, et al. Do multi-hop readers dream of reasoning chains?[C]//Proceedings of the 2nd Workshop on Machine Reading for Question Answering, 2019: 91-97.
[65] CHEN J,DURRETT G. How to learn (and how not to learn) multi-hop reasoning with memory networks[DB/OL]. https://openreview.net/pdf?id=B1lf43A5Y7[2021-05-07].
[66] SHAO N, CUI Y, LIU T, et al. Is graph structure necessary for multi-hop question answering?[C]//Proceedings of the Conference on Empirical Methods in Natural Language Processing. Online: Association for Computational Linguistics, 2020: 7187-7192.
[67] USTER S, SUSHIL M,DAELEMANS W. Why cant memory networks read effectively?[J/OL]. arXiv preprint arXiv: 1910.07350, 2019.
[68] WELBL J, STENETORP P,RIEDEL S. Constructing datasets for multi-hop reading comprehension across documents[J]. Transactions of the Association for Computational Linguistics, 2018, 6: 287-302.
[69] CHEN J,DURRETT G. Understanding dataset design choices for multi-hop reasoning[C]//Proceedings of the North American Chapter of the Association for Computational Linguistics, 2019: 4026-4032.
[70] WESTON J, BORDES A, CHOPRA S, et al. Towards ai-complete question answering: a set of prerequisite toy tasks[C]//Proceedings of the International Conference on Learning Representations, 2016.
[71] CHEN W, ZHA H, CHEN Z, et al. HybridQA: A dataset of multi-hop question answering over tabular and textual data[C]//Proceedings of the Findings of the Association for Computational Linguistics: EMNLP, 2020: 1026-1036.
[72] TALMOR A,BERANT J. The web as a knowledge-base for answering complex questions[C]//Proceedings of the Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, New Orleans, Louisiana, USA: Association for Computational Linguistics, 2018: 641-651.
[73] ABUJABAL A, ROY R S, YAHYA M, et al. ComQA: A community-sourced dataset for complex factoid question answering with paraphrase clusters[C]//Proceedings of the Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. Minneapolis, MN, USA: Association for Computational, 2019: 307-317.
[74] KHOT T, CLARK P, GUERQUIN M, et al. QASC: A dataset for question answering via sentence composition[C]//Proceedings of the 34th AAAI Conference on Artificial Intelligence, 2020.
[75] WOLFSON T, GEVA M, GUPTA A, et al. Break it down: A question understanding benchmark[J]. Transactions of the Association for Computational Linguistics, 2020, 8: 183-198.
[76] KOCˇISKY＇ T, SCHWARZ J, BLUNSOM P, et al. The narrativeqa reading comprehension challenge[J]. Transactions of the Association for Computational Linguistics, 2018, 6: 317-328.
[77] KHASHABI D, CHATURVEDI S, ROTH M, et al. Looking beyond the surface: A challenge set for reading comprehension over multiple sentences[C]//Proceedings of the Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. New Orleans, Louisiana, USA: Association for Computational Linguistics, 2018: 252-262.
[78] DUA D, WANG Y, DASIGI P, et al. DROP: A reading comprehension benchmark requiring discrete reasoning over paragraphs[C]//Proceedings of the Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Minneapolis, MN, USA: Association for Computational, 2019: 2368-2378.
[79] REDDY S, CHEN D,MANNING C D. COQA: A conversational question answering challenge[J]. Transactions of the Association for Computational Linguistics, 2019, 7: 249-266.
[80] CHOI E, HE H, IYYER M, et al. QUAC: Question answering in context[C]//Proceedings of the Conference on Empirical Methods in Natural Language Processing, 2018: 2174-2184.
[81] ELGOHARY A, ZHAO C,BOYD-GRABER J. a dataset and baselines for sequential open-domain question answering[C]//Proceedings of the Conference on Empirical Methods in Natural Language Processing, 2018: 1077-1083.
[82] MIHAYLOV T, CLARK P, KHOT T, et al. Can a suit of armor conduct electricity?: A new dataset for open book question answering[C]//Proceedings of the Conference on Empirical Methods in Natural Language Processing, 2018: 2381-2391.
[83] SAHA A, ARALIKATTE R, KHAPRA M M, et al. DuoRC: Towards complex language understanding with paraphrased reading comprehension[C]//Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics. Melbourne, Australia: Association for Computational Linguistics, 2018: 1683-1693.
[84] SEO M, KEMBHAVI A, FARHADI A, et al. Bidirectional attention flow for machine comprehension[C]//Proceedings of the International Conference on Learning Representations, 2017.
[85] DEVLIN J, CHANG M W, LEE K, et al. BERT: Pretraining of deep bidirectional transformers for language understanding[C]//Proceedings of the Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. Minneapolis, MN, USA: Association for Computational, 2019: 4171-4186.
[86] TALMOR A,BERANT J. MultiQA: An empirical investigation of generalization and transfer in reading comprehension[C]//Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, 2019: 4911-4921.
[87] WEISSENBORN D, WIESE G,SEIFFE L. Making neural QA as simple as possible but not simpler[C]//Proceedings of the 21st Conference on Computational Natural Language Learning, 2017: 271-280.
[88] DHINGRA B, JIN Q, YANG Z, et al. Neural models for reasoning over multiple mentions using coreference[C]//Proceedings of the Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. New Orleans, Louisiana, USA: Association for Computational Linguistics, 2018: 42-48.
[89] RADFORD A, WU J, CHILD R, et al. Language models are unsupervised multitask learners[DB/OL]. https://cdn.openai.com/better-language-models/language_models_are_unsupervised_multitask_learners.pdf[2021-05-07].
[90] YANG H, WANG H, GUO S, et al. Learning to decompose compound questions with reinforcement learning[DB/OL]. https://openreview.net/pdf?id=SJl2ps0qKQ [2021-05-07].
|
|
|
|
2022, Vol. 36 
