基于深度学习的多文档机器阅读理解综述

高峰,倪建成,高鹏,周子力,李艳艳

PDF(3984 KB)
PDF(3984 KB)
中文信息学报 ›› 2023, Vol. 37 ›› Issue (8) : 1-17.
综述

基于深度学习的多文档机器阅读理解综述

  • 高峰1,2,倪建成3,高鹏1,周子力1,李艳艳1
作者信息 +

Survey of Multi-document Machine Reading Comprehension Based on Deep Learning

  • GAO Feng1,2, NI Jiancheng3, GAO Peng1, ZHOU Zili1, LI Yanyan1
Author information +
History +

摘要

多文档机器阅读理解是利用计算机同时将多个文档的语义信息进行阅读理解、筛选糅合后进行问题作答的过程。与传统的单文档机器阅读理解相比,其更关注问题约束下的文档间语义逻辑关系理解,是自然语言处理领域中新兴的研究方向之一。该文首先选取了多文档机器阅读理解的重点数据集,涵盖多源信息型数据集、线性推理型数据集和细粒度半结构化型数据集等以分析任务发展趋势;其次调研了深度学习领域中主流的评价指标和研究方法,划分了具有代表性的四层阅读架构并分别展开了详细的讨论,特别在阅读理解层中选取了文档选择、Transformer语言模型、图卷积神经网络和外部知识融合等阅读方法以分析任务的研究现状;最后对多文档阅读理解的研究进行了总结与展望,提出了后续研究面临的问题与挑战。

Abstract

Multi-document machine reading comprehension requires the machine to read and comprehend multi documents simultaneously, so as to answer subsequent questions. Compared with the traditional single-document machine reading comprehension, it is more concerned with the semantic logic among the documents within the constraints of questions. The paper firstly explores the main datasets for multi-document machine reading comprehension to analyze its development trends, including multi-source datasets, linear inference datasets, and fine-grained semi-structured datasets. Secondly, it investigates the mainstream evaluation metrics, then a prototype of four-tier reading structure is discussed in detail. This paper also reviews the methods based on document selection, Transformers, Graph Convolution Networks, and external knowledge fusion. Finally, it gives an outlook on the research of multi-document reading comprehension, as well as the problems and challenges in this study.

关键词

综述 / 多文档机器阅读理解 / 深度学习

Key words

survey / multi-document machine reading comprehension / deep learning

引用本文

EndNote

Ris (Procite)

Bibtex

导出引用
高峰,倪建成,高鹏,周子力,李艳艳. 基于深度学习的多文档机器阅读理解综述. 中文信息学报. 2023, 37(8): 1-17
GAO Feng, NI Jiancheng, GAO Peng, ZHOU Zili, LI Yanyan. Survey of Multi-document Machine Reading Comprehension Based on Deep Learning. Journal of Chinese Information Processing. 2023, 37(8): 1-17

参考文献

[1] ADAMS M, BRUCE B. Background knowledge and reading comprehension[J]. Reader Meets Author: Bridging the Gap, 1982, 13: 2-25.
[2] CHEN D. Neural reading comprehension and beyond[D]. PhD Diss., Stanford University, 2018.
[3] 苏立新, 郭嘉丰, 范意兴, 等. 面向多片段答案的抽取式阅读理解模型[J]. 计算机学报, 2020, 43(05): 856-867.
[4] DEVLIN J, CHANG M, LEE K, et al. BERT: Pre-training of deep bidirectional transformers for language understanding[C]//Proceedings of the Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, 2019: 4171-4186.
[5] RAJPURKAR P, JIA R, LIANG P. Know what you don't know: Unanswerable questions for SQuAD[C]//Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics, 2018: 784-789.
[6] RAJPURKAR P, ZHANG J, LOPYREV K, et al. SQuAD: 100,000+ Questions for machine comprehension of text[C]//Proceedings of the Conference on Empirical Methods in Natural Language Processing, 2016: 2383-2392.
[7] SUN K, YU D, YU D, et al. Investigating prior knowledge for challenging Chinese machine reading comprehension[J]. Transactions of the Association for Computational Linguistics, 2020, 8: 141-155.
[8] WELBL J, STENETORP P, RIEDEL S. Constructing datasets for multi-hop reading comprehension across documents[J]. Transactions of the Association for Computational Linguistics, 2018, 6: 287-302.
[9] HE W, LIU K, LIU J, et al. DuReader: A Chinese machine reading comprehension dataset from real-world applications[C]//Proceedings of the Workshop on Machine Reading for Question Answering @ ACL, 2018: 37-46.
[10] NGUYEN T, ROSENBERG M, SONG X, et al. MS MARCO: A human generated machine Reading comprehension dataset[C]//Proceedings of the Workshop on Cognitive Computation: Integrating Neural and Symbolic Approaches Co-located with the 30th Annual Conference on Neural Information Processing Systems, 2016.
[11] YANG Z, QI P, ZHANG S, et al. HotpotQA: A dataset for diverse, explainable multi-hop question answering[C]//Proceedings of the Conference on Empirical Methods in Natural Language Processing, 2018: 2369-2380.
[12] 王军平, 张文生, 王勇飞, 等. 面向大数据领域的事理认知图谱构建与推断分析[J]. 中国科学(信息科学), 2020, 50(07): 988-1002.
[13] BARADARAN R, GHIASI R, AMIRKHANI H. A survey on machine reading comprehension systems[J]. Natural Language Engineering, 2022, 28(6): 683-732.
[14] JOSHI M, CHOI E, WELD D, et al. TriviaQA: A large scale distantly supervised challenge dataset for reading comprehension[C]//Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics, 2017: 1601-1611.
[15] COHEN K B, HUNTER L. Natural language processing and systems biology[M]. Artificial Intelligence Methods and Tools for Systems Biology. Dordrecht: Springer Netherlands, 2004: 147-173.
[16] LAW V, KNOX C, DJOUMBOU Y, et al. DrugBank 4.0: Shedding new light on drug metabolism[J]. Nucleic Acids Research, 2014, 42: 1091-1097.
[17] WISHART D S, FEUNANG Y D, GUO A C, et al. DrugBank 5.0: A major update to the DrugBank database for 2018[J]. Nucleic Acids Res, 2018, 46(D1): D1074-D1082.
[18] GUPTA M, KULKARNI N, CHANDA R, et al. AmazonQA: A review-based question answering task[C]//Proceedings of the 28th International Joint Conference on Artificial Intelligence, 2019: 4996-5002.
[19] INOUE N, STENETORP P, INUI K. R4C: A benchmark for evaluating rc systems to get the right answer for the right reason[C]//Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, 2020: 6740-6750.
[20] ONISHI T, WANG H, BANSAL M, et al. Who did what: A large-scale person-centered cloze dataset[C]//Proceedings of the Conference on Empirical Methods in Natural Language Processing, 2016: 2230-2235.
[21] BERZAK Y, MALMAUD J, LEVY R. STARC: structured annotations for reading comprehension[C]//Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, 2020: 5726-5735.
[22] KHASHABI D, CHATURVEDI S, ROTH M, et al. Looking beyond the surface: A challenge set for reading comprehension over multiple sentences[C]//Proceedings of the Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, 2018: 252-262.
[23] PAPINENI K, ROUKOS S, WARD T, et al. Bleu: A method for automatic evaluation of machine translation[C]//Proceedings of the 40th Annual Meeting of the Association for Computational Linguistics, 2002: 311-318.
[24] LIN C Y. ROUGE: A Package for Automatic Evaluation of Summaries[G]//Text Summarization Branches Out. Barcelona, Spain: Association for Computational Linguistics, 2004: 74-81.
[25] 李金鹏, 张闯, 陈小军, 等. 自动文本摘要研究综述[J]. 计算机研究与发展, 2021, 58(1): 1-21.
[26] KIM Y. Convolutional neural networks for sentence classification[C]//Proceedings of the Conference on Empirical Methods in Natural Language Processing, 2014: 1746-1751.
[27] DHINGRA B, ZHOU Z, FITZPATRICK D, et al. Tweet2Vec: Character-based distributed representations for social media[C]//Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics, 2016: 269-274.
[28] SEO M, KEMBHAVI A, FARHADI A, et al. Bidirectional attention flow for machine comprehension[C]//Proceedings of the 5th International Conference on Learning Representations, 2017.
[29] HASHIMOTO K, XIONG C, TSURUOKA Y, et al. A joint many-task model: Growing a neural network for multiple NLP tasks[C]//Proceedings of the Conference on Empirical Methods in Natural Language Processing, 2017: 1923-1933.
[30] ARORA M, KANSAL V. Character level embedding with deep convolutional neural network for text normalization of unstructured data for twitter sentiment analysis[J]. Social Network Analysis and Mining, 2019, 9(1): 1211-1214.
[31] NISHIDA K, NISHIDA K, NAGATA M, et al. Answering while summarizing: Multi-task learning for multi-hop QA with evidence extraction[C]//Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, 2019: 2335-2345.
[32] CLARK C, GARDNER M. Simple and effective multi-paragraph reading comprehension[C]//Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics, 2018: 845-855.
[33] 顾迎捷, 桂小林, 李德福, 等. 基于神经网络的机器阅读理解综述[J]. 软件学报, 2020, 31(07): 2095-2126.
[34] TU M, WANG G, HUANG J, et al. Multi-hop reading comprehension across multiple documents by reasoning over heterogeneous graphs[C]//Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, 2019: 2704-2713.
[35] MIKOLOV T, CHEN K, CORRADO G, et al. Efficient estimation of word representations in vector space[C]//Proceedings of the 1st International Conference on Learning Representations, 2013.
[36] LE Q V, MIKOLOV T. Distributed representations of sentences and documents[C]//Proceedings of the 31th International Conference on Machine Learning, 2014: 1188-1196.
[37] PENNINGTON J, SOCHER R, MANNING C D. Glove: Global vectors for word representation[C]//Proceedings of the Empirical Methods in Natural Language Processing, 2014: 1532-1543.
[38] TANG Z, SHEN Y, MA X, et al. Multi-hop reading comprehension across documents with path-based graph convolutional network[C]//Proceedings of the 29th International Joint Conference on Artificial Intelligence, 2020: 3905-3911.
[39] CAO Y, FANG M, TAO D. BAG: Bi-directional attention entity graph convolutional network for multi-hop reasoning question answering[C]//Proceedings of the Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, 2019: 357-362.
[40] PETERS M E, NEUMANN M, IYYER M, et al. Deep contextualized word representations[C]//Proceedings of the Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, 2018: 2227-2237.
[41] BROWN T B, MANN B, RYDER N, et al. Language models are few-shot learners[C]//Proceedings of the Annual Conference on Neural Information Processing Systems, 2020.
[42] LEWIS P S H, PEREZ E, PIKTUS A, et al. Retrieval-augmented generation for knowledge-intensive NLP tasks[C]//Proceedings of the Annual Conference on Neural Information Processing Systems, 2020.
[43] WILLIAMS R J, ZIPSER D. A learning algorithm for continually running fully recurrent neural networks[J]. Neural Computation, 1989, 1(2): 270-280.
[44] HOCHREITER S, SCHMIDHUBER J. Long short-term memory[J]. Neural Computation, 1997, 9(8): 1735-1780.
[45] CHUNG J, GULCEHRE C, CHO K, et al. Empirical evaluation of gated recurrent neural networks on sequence modeling[J/OL]. arXiv preprint arXiv:1412.3555, 2014.
[46] WANG Y, LIU K, LIU J, et al. Multi-passage machine reading comprehension with cross-passage answer verification[C]//Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics, 2018: 1918-1927.
[47] YAN M, XIA J, WU C, et al. A deep cascade model for multi-document reading comprehension[C]//Proceedings of the 33rd AAAI Conference on Artificial Intelligence, 2019: 7354-7361.
[48] KUNDU S, NG H T. A question-focused multi-factor attention network for question answering[C]//Proceedings of the 32nd AAAI Conference on Artificial Intelligence, 2018: 5828-5835.
[49] WEISSENBORN D, KOISK T, Dyer C. Dynamic integration of background knowledge in neural NLU systems[J/OL]. arXiv preprint arXiv:1706.02596, 2017.
[50] JIANG Y, JOSHI N, CHEN Y C, et al. Explore, propose, and assemble: An interpretable model for multi-hop reading comprehension[C]//Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, 2019: 2714-2725.
[51] VASWANI A, SHAZEER N, PARMAR N, et al. Attention is all you need[C]//Proceedings of the Annual Conference on Neural Information Processing Systems, 2017: 5998-6008.
[52] NISHIDA K, SAITO I, NISHIDA K, et al. Multi-style generative reading comprehension[C]//Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, 2019: 2273-2284.
[53] AINSLIE J, ONTAN S, ALBERTI C, et al. ETC: Encoding long and structured inputs in transformers[C]//Proceedings of the Conference on Empirical Methods in Natural Language Processing, 2020: 268-284.
[54] ZAHEER M, GURUGANESH G, DUBEY K A, et al. Big Bird: Transformers for longer sequences[C]//Proceedings of the Annual Conference on Neural Information Processing Systems, 2020.
[55] MAO Y, HE P, LIU X,et al. Generation-Augmented Retrieval for Open-Domain Question Answering[C]//Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing, 2021: 4089-4100.
[56] IZACARD G, GRAVE E. Leveraging Passage Retrieval with Generative Models for Open Domain Question Answering[C]//Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics: Main Volume. 2021: 874-880.
[57] BELTAGY I, PETERS M E, COHAN A. Longformer: the long-document transformer[J/OL]. arXiv preprint arXiv:2004.05150, 2020.
[58] LEWIS M, LIU Y, GOYAL N, et al. BART: Denoising sequence-to-sequence pre-training for natural language generation, translation, and comprehension[C]//Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, 2020: 7871-7880.
[59] LIU Z, ZHOU J. Introduction to graph neural networks[J]. Synthesis Lectures on Artificial Intelligence and Machine Learning, 2020, 14(2): 1-127.
[60] CAO N D, AZIZ W, TITOV I. Question answering by reasoning across documents with graph convolutional networks[C]//Proceedings of the Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, 2019: 2306-2317.
[61] SCHLICHTKRULL M, KIPF T N, BLOEM P, et al. Modeling relational data with graph convolutional networks[C]//Proceedings of the European Semantic Web Conference, 2018: 593-607.
[62] DING M, ZHOU C, CHEN Q, et al. Cognitive graph for multi-hop reading comprehension at scale[C]//Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, 2019: 2694-2703.
[63] YE D, LIN Y, LIU Z, et al. Multi-paragraph reasoning with knowledge-enhanced graph neural network[J/OL]. arXiv preprint arXiv:1911.02170,2019.
[64] YANG A, WANG Q, LIU J, et al. Enhancing pre-trained language representations with rich knowledge for machine reading comprehension[C]//Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, 2019: 2346-2357.
[65] SUN H, BEDRAX WEISS T, COHEN W W. PullNet: Open domain question answering with iterative retrieval on knowledge bases and text[C]//Proceedings of the Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing, 2019: 2380-2390.
[66] 谭红叶, 李宣影, 刘蓓. 基于外部知识和层级篇章表示的阅读理解方法[J]. 中文信息学报, 2020, 34(04): 85-91.

基金

曲阜师范大学高层次人才科研基金(602801)
PDF(3984 KB)

Accesses

Citation

Detail
段落导航
相关文章

  • 版权所有 © 《中文信息学报》编辑部 网站统计

    地址：北京市海淀区中关村南四街4号　邮编：100190　电话：010-62562916　E-mail：cips@iscas.ac.cn

    本系统由北京玛格泰克科技发展有限公司设计开发

/

〈 〉