实体关系抽取是实现海量文本数据知识化、自动构建大规模知识图谱的关键技术。考虑到头尾实体信息对关系抽取有重要影响,该文采用注意力机制将实体对信息融合到关系抽取过程中,提出了基于实体对注意力机制的实体关系联合抽取模型(EPSA)。首先,使用双向长短时记忆网络(Bi-LSTM)结合条件随机场(CRF)完成实体的识别;其次,将抽取的实体配对,信息融合成统一的嵌入式表示形式,用于计算句子中各词的注意力值;然后,使用基于实体对注意力机制的句子编码模块得到句子表示,再利用显式融合实体对的信息得到增强型句子表示;最后,通过分类方式完成实体关系的抽取。在公开数据集NYT和WebNLG上对提出的EPSA模型进行评估,实现结果表明,与目前主流联合抽取模型相比,EPSA模型在F1值上均得到提升,分别达到84.5%和88.5%,并解决了单一实体重叠问题。
Abstract
Entity and relation extraction is a key technology to automatically build large-scale knowledge graphs from massive text data. Considering the effect of the entity on the discrimination of relation types, this paper proposes a joint entity and relation extraction model based on entity-pair specific attention mechanism (EPSA). First, the entity recognition is completed based on Bi-directional Long Short-Term Memory (Bi-LSTM) and Conditional Random Fields (CRF). Then the extracted entities are combined into entity-pairs and transformed into a unified embedding. The sentence representation is obtained by the entity-pair specific attention mechanism plus the entity-pair embedding. And finally, the relation extraction is completed by the a classification process. Experimental results on NYT and WebNLG datasets show that the proposed method out-performs the baselines by achieving 84.5% and 88.5% F1 value, respectively.
关键词
知识图谱 /
注意力机制 /
实体关系联合抽取
{{custom_keyword}} /
Key words
knowledge graph /
attention mechanism /
joint entity and relation extraction
{{custom_keyword}} /
{{custom_sec.title}}
{{custom_sec.title}}
{{custom_sec.content}}
参考文献
[1] 王传栋, 徐娇, 张永.实体关系抽取综述[J].计算机工程与应用,2020,56(12): 25-36.
[2] 刘知远, 孙茂松, 林衍凯, 等. 知识表示学习研究进展[J]. 计算机研究与发展, 2016, 53(2): 247-261.
[3] 刘峤, 李杨, 段宏, 等. 知识图谱构建技术综述[J]. 计算机研究与发展, 2016, 53(3): 582-600.
[4] 李冬梅, 张扬, 李东远, 等. 实体关系抽取方法研究综述[J]. 计算机研究与发展,2020, 57(7): 1424-1448.
[5] 江开忠, 李子成, 顾君忠. 自动文本摘要方法[J]. 计算机工程, 2008, 34(1): 221-223.
[6] 刘里, 曾庆田. 自动问答系统研究综述[J]. 山东科技大学学报(自然科学版), 2007, 26(04): 73-76.
[7] ATIF F, EL KHATIB O, DIFALLAH D. Beamqa: Multi-hop knowledge graph question answering with sequence-to-sequence prediction and beam search[C]//Proceedings of the 46th International ACM SIGIR Conference on Research and Development in Information Retrieval. 2023: 781-790.
[8] 邓擘, 樊孝忠, 杨立公. 用语义模式提取实体关系的方法[J]. 计算机工程, 2007(10): 212-214.
[9] 刘方驰, 钟志农, 雷霖, 等. 基于机器学习的实体关系抽取方法[J]. 兵工自动化, 2013, 32(09): 57-62.
[10] 庄传志, 靳小龙, 朱伟建, 等. 基于深度学习的关系抽取研究综述[J]. 中文信息学报, 2019, 33(12): 1-18.
[11] LIU C Y, SUN W B, CHAO W H, et al. Convolution neural network for relation extraction[C]//Proceedings of the Int Conference on Advanced Data Mining and Applications. Berlin: Springer, 2013: 231-242.
[12] ZENG D J, LIU K, LAI S W, et al. Relation classification via convolutional deep neural network[C]//Proceedings of the 25th Int Conference on Computational Linguistics. Stroudsburg, PA: ACL, 2014: 2335-2344.
[13] SOCHER R, HUVAL B, MANNING C D, et al. Semantic compositionality through recursive matrix-vector spaces[C]//Proceedings of the Joint Conference on Empirical Methods in Natural Language Processing and Computational Natural Language Learning. Stroudsburg: ACL, 2012: 1201-1211.
[14] XU Y, MOU L L, LI G, et al. Classifying relations via long short term memory networks along shortest dependency paths[C]//Proceedings of the Conference on Empirical Methods in Natural Language Processing. Stroudsburg, PA: ACL, 2015: 1785-1794.
[15] 鄂海红, 张文静, 肖思琪,等. 深度学习实体关系抽取研究综述 [J].软件学报, 2019, 30(6): 1793-1818.
[16] MIWA M, BANSAL M. End-to-end relation extraction using LSTMs on sequences and tree structures[C]//Proceedings of the 54th Annual Meeting of the ACL. Stroudsburg, PA: ACL, 2016: 1105-1116.
[17] ZHENG S C, WANG F, BAO H Y, et al. Joint extraction of entities and relations based on a novel tagging scheme[C]//Proceedings of the 55th Annual Meeting of the ACL. Stroudsburg, PA: ACL, 2017: 1227-1236.
[18] ZENG X R, ZENG D, HE S Z, et al. Extracting relational facts by an end-to-end neural model with copy mechanism[C]//Proceedings of the 56th Annual Meeting of the ACL, 2018: 506-514.
[19] ZENG D J, ZHANG H R, LIU Q Y. CopyMTL: Copy mechanism for joint extraction of entities and relations with multi-task learning[C]//Proceedings of the 34th AAAI Conference on Artificial Intelligence. Palo Alto, CA: AAAI, 2020: 9507-9514.
[20] FU T J, LI P, MA W Y. GraphRel: Modeling text as relational graphs for joint entity and relation extraction[C]//Proceedings of the 57th Annual Meeting of the ACL, 2019: 1409-1418.
[21] DUAN G, MIAO J, HUANG T, et al. A relational adaptive neural model for joint entity and relation extraction[J]. Frontiers in Neurorobotics, 2021: 15: 635492.
[22] YUAN Y, ZHOU X F, PAN S R, et al. A relation-specific attention network for joint entity and relation extraction[C]//Proceedings of the 29th International Joint Conference on Artificial Intelligence. IJCAI, 2020: 4054-4060.
[23] YU B, ZHANG Z, SHU X, et al. Joint extraction of entities and relations based on a novel decomposition strategy[J]. arXiv preprint arXiv: 1909.04273, 2019.
[24] ZHONG Z X, CHEN D Q. A frustratingly easy approach for entity and relation extraction[J]. arXiv preprint arXiv: 2010.12812, 2020.
[25] HOCHREITER S, SCHMIDHUBER J. Long short-term memory[J]. Neural Computation, 1997, 9(8): 1735-1780.
[26] LAFFERTY D J, MCCALLUM A, FERNANDO C N P. Conditional random fields: probabilistic models for segmenting and labeling sequence data[C]//Proceedings of the 18th Int Conference on Machine Learning, 2001: 282-289.
[27] VASWANI A, SHAZEER N, PARMAR N, et al. Attention is all you need[J].arXiv. Preeprint arXiv, 2017.
[28] RIEDELS, YAO L M, MCCALLUM A. Modeling relations and their mentions without labeled text[C]//Proceedings of Joint European Conference on Machine Learning and Knowledge Discovery in Databases, 2010: 148-163.
[29] CLAIRE G, ANASTASIA S, SHASHI N, et al. Creating training corpora for NLG micro-planners[C]//Proceedings of the 55th Annual Meeting of the ACL, 2017: 179-188.
[30] ZENG X, HE S, ZENG D, et al. Learning the extraction order of multiple relational facts in a sentence with reinforcement learning[C]//Proceedings of the Empirical Methods in Natural Language Processing. Association for Computational Linguistics, 2019.
[31] 乔勇鹏, 于亚新, 刘树越, 等. 图卷积增强多路解码的实体关系联合抽取模型[J/OL]. 计算机研究与发展: 1-14. http://kns.cnki.net/kcms/detail/11.1777.TP.20220506.1812.008.html[2022-06-30]
{{custom_fnGroup.title_cn}}
脚注
{{custom_fn.content}}
基金
国家自然科学基金(61972266)
{{custom_fund}}
PDF(3133 KB)
