基于图神经网络和语义知识的自然语言推理任务研究

摘要
参考文献
相关文章 (15)

全文: PDF (4134 KB) HTML (1 KB)
输出: BibTeX | EndNote (RIS)

摘要自然语言推理任务的目的是推断两个句子之间的语义逻辑关系。该文通过模仿人类的推理过程构造模型,首先利用长短时记忆网络提取词的语境特征,模仿人类粗读句子的过程;然后依据外部语义知识,连接两个句子中有语义联系的词,构造一个以词为节点的语义图;接下来模仿人类比较两个句子的语义角色相似性的思维,用图卷积或图注意力神经网络聚合词在图中的空间特征;最后融合词的语境特征和语义图空间特征,进行推理分类。实验结果证明,基于图神经网络的模型能有效利用外部语义知识来提高自然语言推理的准确率。

	服务

	把本文推荐给朋友
	加入我的书架
	加入引用管理器
	E-mail Alert
	RSS
	作者相关文章
	刘欣瑜
	刘瑞芳
	石航
	韩斌

关键词 ：自然语言推理, 图神经网络, 语义知识, 双向长短时记忆网络

Abstract：Natural language inference is to infer the semantic logical relationship between two given sentences. This paper proposes an inference model to simulate human thinking. Firstly, the context features of sentences are extracted by BiLSTM (bidirectional long short-term memory), which imitates human beings to understand sentence meaning. Then, the semantic graph for every pair of sentences is constructed according to the external semantic knowledge. The spatial features of words are extracted by graph convolutional network or graph attention network, which simulates the thinking mode of analyzing the semantic role similarity of two sentences. Finally, the semantic relationship of two sentences is inferred by integrating the context features and the spatial features. Further analysis reveals that the semantic knowledge is better exploited by graph neural network in natural language inference task.

Key words： natural language inference graph neural network semantic knowledge Bi-LSTM

收稿日期: 2020-11-10

中图分类号:

TP391

作者简介: 刘欣瑜(1995—),硕士研究生,主要研究领域为自然语言处理。E-mail: liuxy@bupt.edu.cn
刘瑞芳(1972—),通信作者,博士,副教授,主要研究领域为自然语言处理。E-mail: lrf@bupt.edu.cn
石航(1995—),硕士研究生,主要研究领域为自然语言处理。E-mail: sh45@bupt.edu.cn

链接本文:

http://jcip.cipsc.org.cn/CN/Y2021/V35/I6/122

[1] Zhou J, Cui G, Zhang Z, et al. Graph neural networks: A review of methods and applications[J/OL]. arXiv preprint arXiv:1812.08434, 2018.
[2] Bowman S R,Angeli G, Potts C, et al. A large annotated corpus for learning natural language inference [C]//Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing, 2015: 632-642.
[3] Williams A,Nangia N, Bowman S. A broad-coverage challenge corpus for sentence understanding through inference [C]//Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, 2018.
[4] Rocktschel T, Grefenstette E, Hermann K M, et al. Reasoning about entailment with neural attention [C]//Proceedings of the 4th International Conference on Learning Representations, 2016.
[5] Chen Q, Zhu X, Ling Z, et al. Enhanced LSTM for natural language inference [C]//Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics, 2016.
[6] Chen Q, Zhu X, Ling Z H, et al. Neural natural language inference models enhanced with external knowledge [C]//Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics, 2018.
[7] Fellbaum C, Miller G. WordNet: An electronic lexical database [M].WordNet: An Electronic Lexical Database. MIT Press, 1998.
[8] 袁毓林,卢达威. 怎样利用语言知识资源进行语义理解和常识推理[J]. 中文信息学报, 2018, 32(12): 11-23.
[9] 谭咏梅,刘姝雯,吕学强. 基于CNN与双向LSTM的中文文本蕴含识别方法[J]. 中文信息学报, 2018, 32(7): 11-19.
[10] Marcheggiani D, Titov I. Encoding sentences with graph convolutional networks for semantic role labeling [C]//Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, 2017.
[11] Vashishth S, Bhandari M, Yadav P, et al. Incorporating syntactic and semantic information in word embeddings using graph convolutional networks[C]//Proceedings of the 57th Conference of the Association for Computational Linguistics, 2019: 3308-3318.
[12] Sahu S K, Christopoulou F, Miwa M, et al. Inter-sentence relation extraction with document-level graph convolutional neural network[C]//Proceedings of the 57th Conference of the Association for Computational Linguistics, 2019: 4309-4316.
[13] Velikovi P, Cucurull G, Casanova A, et al. Graph attention networks[C]//Proceedings of the 6th International Conference on Learning Representations, 2017.
[14] Zhang Z, Wu Y, Zhao H, et al. Semantics-aware BERT for language understanding[J]. Proceedings of the AAAI Conference on Artificial Intelligence, 2020, 34(5):9628-9635.
[15] Palmer M, Gildea D, Kingsbury P. The proposition bank: An annotated corpus of semantic roles[J]. Computational Linguistics, 2005, 31(1):71-106.
[16] Wang S, Jiang J. Learning natural language inference with LSTM [C]//Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, 2016.
[17] Cheng J, Dong L,Lapata M. Long short-term memory: Networks for machine reading [C]//Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing, 2016: 551-561.
[18] Chen Q, Zhu X, Ling Z H, et al. Recurrent neural network-based sentence encoder with gated attention for natural language inference[C]//Proceedings of the 2nd Workshop on Evaluating Vector Space Representations for NLP, 2017.
[19] Huang G, Liu Z, Laurens V D M, et al. Densely connected convolutional networks[J]//Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition, 2016: 2261-2269.
[20] Kim S, Kang I, Kwak N. Semantic sentence matching with densely-connected recurrent and co-attentive information[J]. Proceedings of the AAAI Conference on Artificial Intelligence, 2019, 33: 6586-6593.