基于异构图神经网络的高考阅读理解问答研究

杨陟卓,李沫谦,张虎,李茹

PDF(2927 KB)
PDF(2927 KB)
中文信息学报 ›› 2023, Vol. 37 ›› Issue (5) : 101-111.
问答与对话

基于异构图神经网络的高考阅读理解问答研究

  • 杨陟卓1,李沫谦1,张虎1,李茹1,2
作者信息 +

Question Answering in Reading Comprehension of College Entrance Examination Based on Heterogeneous Graph Neural Network

  • YANG Zhizhuo1, LI Moqian1, ZHANG Hu1, LI Ru1,2
Author information +
History +

摘要

机器阅读理解是自然语言处理领域的核心任务,高考阅读理解自动问答是近年来阅读理解任务中的重要挑战。由于高考题难度较大,同时高考阅读理解问答的数据集较少,导致传统的方法答题效果欠佳。基于此,该文提出一种基于异构图神经网络的答案句抽取模型,将丰富的节点(句子节点、词语节点)和节点之间的关系(框架关系、篇章主题关系)引入图神经网络模型中,问句不仅可以通过中继词语节点与候选句节点进行交互,还可以通过框架语义和篇章主题关系与候选节点进行相互更新。不同类型的语义节点和多维度的语义关系可以帮助模型更好地对信息进行筛选、理解和推理。模型在北京高考语文真题上进行测试,实验结果表明,基于图神经网络的问答模型答题效果优于基线模型,F1值达到了78.08%,验证了该方法的有效性。

Abstract

The question answering of college entrance examination reading comprehension is an important challenge in reading comprehension task in recent years. This paper proposes a model of answer sentence extraction based on heterogeneous graph neural network. Rich relationships (frame semantics and discourse topic relationships ) between nodes (sentences and words ) are introduced into the graph neural network. Therefore, questions can interact with candidate answer sentences through both words nodes and frame semantics and discourse topic relationships. The results show that the proposed model outperforms the baseline model with 78.08% F1 value.

关键词

阅读理解问答 / 答案句抽取 / 异构图神经网络 / 框架语义 / 篇章主题

Key words

reading comprehension QA / answer sentence extraction / heterogeneous graph neural network / frame semantics / discourse topic

引用本文

导出引用
杨陟卓,李沫谦,张虎,李茹. 基于异构图神经网络的高考阅读理解问答研究. 中文信息学报. 2023, 37(5): 101-111
YANG Zhizhuo, LI Moqian, ZHANG Hu, LI Ru. Question Answering in Reading Comprehension of College Entrance Examination Based on Heterogeneous Graph Neural Network. Journal of Chinese Information Processing. 2023, 37(5): 101-111

参考文献

[1] FUJITA A, KAMEDA A, KAWAZOE A, et al. Overview of todai robot project and evaluation framework of its nlp-based problem solving[C]//Proceedings of the 9th International Conference on Language Kesources and Evaluation, 2014: 2590-2597.
[2] HERMANN K M, KOCISKY T, GREFENSTETTE E, et al. Teaching machines to read and comprehend[C]//Proceedings of the 28th International Conference on Neural Information Processing Systems, 2015: 1693-1701.
[3] CUI Y, LIU T, CHEN Z, et al. Consensus attention-based neural networks for Chinese reading comprehension[C]//Proceeding of the 26th International Conference on Computational Linguistics. Japan, 2016: 1777-1786.
[4] RICHARDSON M, BURGES C J C, RENSHAW E. MCTest: a challenge dataset for the open domain machine comprehension of text[C]//Proceeding of the Conference on Empirical Methods in Natural Language Processing. Seattle, Washington, USA: Association for Computational Linguistics, 2013: 193-203.
[5] RODRIGO A, PENAS A, MIYAO Y, et al. Overview of CLEF QA entrance exams task 2015[C]//Proceeding of the CLEF, 2015: 59-99.
[6] PENAS A, MIYAO Y, RODRIGO A, et al. Overview of CLEF QA entrance exams task 2014[C]//Proceeding of the CLEF, 2014: 1194-1200.
[7] RAJPURKAR P, ZHANG J, LOPYREV K, et al. Squad: 100, 000 + questions for machine comprehension of text[C]//Proceedings of the Conference on Empirical Methods in Natural Language Processing, Texas, 2016: 2383-2392.
[8] BAJAJ P, CAMPOS D, CRASWELL N, et al. Ms marco: A human generated machine reading comprehension dataset[C]//Proceedings of the 30th Conference on Neural Information Processing Systems,2016: 1-1.
[9] HE W, LIU K, LIU J, et al. DuReader: A Chinese machine reading comprehension dataset from real-world applications[C]//Proceedings of the Workshop on Machine Reading for Question Answering, 2018: 37-46.
[10] WANG D, NYBERG E. A long short-term memory model for answer sentence selection in question answering[C]//Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing, 2015: 707-712.
[11] 谭红叶, 刘蓓, 王元龙. 基于QU-NNs的阅读理解描述类问题的解答[J].中文信息学报, 2019, 33(3): 102-109.
[12] DEVLIN J, CHANG M W, LEE K, et al. BERT: Pre-training of deep bidirectional transformers for language understanding[C]//Proceedings of the Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, 2019: 4171-4186.
[13] 王智强, 李茹, 梁吉业, 等. 基于汉语篇章框架语义分析的阅读理解问答研究[J]. 计算机学报, 2016, 39(04): 795-807.
[14] GONG Y, BOWMAN S. Ruminating reader: reasoning with gated multi-hop attention[C]//Proceedings of the Workshop on Machine Reading for Question Answering, 2018: 1-11.
[15] YANG L, AI Q, GUO J, et al. aNMM: Ranking short answer texts with attention-based neural matching model[C]//Proceedings of the 25th ACM International on Conference on Information and Knowledge Management. New York, NY, United States: Association for Computing Machinery, 2016: 287-296.
[16] WANG S, JIANG J. A compare-aggregate model for matching text sequences[J]. arXiv preprint arXiv: 1611.01747, 2016.
[17] 卓越. 基于深度学习的智能问答系统的研究与实现[D]. 沈阳: 沈阳师范大学硕士学位论文, 2018.
[18] LI P, LI W, HE Z, et al. Dataset and neural recurrent sequence labeling model for open-domain factoid question answering[J]. arXiv preprint arXiv: 1607.06275, 2016.
[19] 栾克鑫. 基于深度学习的答案融合方法研究[D]. 哈尔滨: 哈尔滨工业大学硕士学位论文, 2017.
[20] 苏立新, 郭嘉丰, 范意兴, 等. 面向多片段答案的抽取式阅读理解模型[J]. 计算机学报, 2020, 43(05): 856-867.
[21] ZHANG Z, ZHAO H. One-shot learning for question-answering in Gaokao history challenge[C]//Proceedings of the 27th International Conference on Computational Linguistics., 2018.
[22] 李国臣, 刘姝林, 杨陟卓, 等. 基于框架语义的高考语文阅读理解答案句抽取[J]. 中文信息学报, 2016, 30(6): 164-172.
[23] 杨陟卓, 李春转, 张虎, 等. 基于CFN和篇章主题的概括型问答题的解答[J]. 中文信息学报, 2020, 34(12): 73-82.
[24] 徐冰冰, 岑科廷, 黄俊杰, 等. 图卷积神经网络综述[J].计算机学报, 2020, 43(05): 755-780.
[25] 杨善良, 常征. 基于图注意力神经网络的中文隐式情感分析[J]. 计算机工程与应用, 2021: 57(24): 161-167.
[26] 刘勋. 基于高低阶图卷积网络的文本分类算法研究[D]. 广州: 广东技术师范大学硕士学位论文, 2020.
[27] 王磊, 熊于宁, 李云鹏, 等. 基于增强图卷积神经网络的协同推荐模型[J]. 计算机研究与发展, 2020, 1-10.
[28] WANG X, HE X, CAO Y, et al. Kgat: Knowledge graph attention network for recommendation[C]//Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. New York, NY, United States: Association for Computing Machinery, 2019: 950-958.
[29] YASUNAGA M, ZHANG R, MEELU K, et al. Graph-based neural multi-document summarization[C]//Proceedings of the 21st Conference on Computational Natural Language Learning, 2017: 452-462.
[30] XU J, DURRETT G. Neural extractive text summarization with syntactic compression[C]//Proceedings of the EMNLP-IJCNLP 2019: 3292-3303.
[31] KIPF T N, WELLING M. Semi-supervised classification with graph Convolutional networks[C]//Proceedings of the 5th International Conference on Learning Representations, ICLR, 2017: 1-14.
[32] VELIC^KOVIC′ P, CUCURULL G, CASANOVA A, et al. Graph Attention Networks[C]//Proceedings of the ICLR, 2018: 1-12.
[33] KIPF T N, WELLING M. Variational graph auto-encoders[C]//Proceedings of the NIPS, 2016.
[34] LI C, WELLING M, ZHU J, et. al, Graphical generative adversarial networks[C]//Proceedings of the Advances in Neural Information Processing Systems, 2018: 6072-6083.
[35] LV S, GUO D, XU J, et. al. Graph-based reasoning over heterogeneous external knowledge for commonsense question answering[C]//Proceedings of the 34th AAAI Conference on Artificial Intelligence, 2020: 8449-8456.
[36] ASAI A, HASHIMOTO K, HAJISHIRZI H, et al. Learning to retrieve reasoning paths over wikipedia graph for question answering[C]//Proceedings of ICLR,2020: 1-22.
[37] ZHENG B, WEN H, LIANG Y, et al. Document modeling with graph attention networks for multi-grained machine reading comprehension[C]//Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics. 2020: 6708-6718.
[38] DING M, ZHOU C, CHEN Q, et al. Cognitive graph for multi-hop reading comprehension at scale[C]//Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, 2019: 2694-2703.
[39] SONG L, WANG Z, YU M, et al. Exploring graph-structured passage representation for multi-hop reading comprehension with graph neural networks[J]. arXiv preprint arXiv: 1809.02040, 2018.
[40] DEVLIN J, CHANG M W, LEE K, et al. BERT: Pre-training of deep bidirectional transformers for language understanding[C]//Proceedings of the NAACL, 2019: 4171-4186.

基金

国家重点研发计划项目(2018YFB1005103);山西省基础研究计划项目面上基金(20210302123469);国家自然科学基金(62176145)
PDF(2927 KB)

Accesses

Citation

Detail

段落导航
相关文章

/