从非结构化文本中进行实体和关系抽取已经成为自然语言处理的一项关键任务,然而命名实体识别(NER)和关系抽取(RE)两个任务经常被分开考虑,从而丢失了大量的关联信息。鉴于此,该文提出了一种端到端的基于多层感知机SGM模块进行信息过滤的实体关系联合抽取方法。该方法在不引入外部其他复杂特征的情况下获得了丰富的语义,充分利用了实体和关系之间的关联。该文从句子级、词语级和字符级三个级别输入信息,利用SGM模块进行信息提取以获得高效的语义表示,之后利用Span-attention进行融合得到Span的具体表示,最后利用全连接层进行实体和关系的联合抽取。该文使用NYT10和NYT11数据集验证所提方法的有效性。实验结果表明,在NYT10和NYT11数据集上,该文提出的模型在关系抽取任务中的F1值分别达到了70.6%和68.3%,相比于其他模型有较大提升。
Abstract
Extracting entities and relations from unstructured text has become a crucial task in natural language processing. We propose an end-to-end joint entity and relation extraction based on SGM module. In our model, word-level and character-level embeddings are transferred to SGM module to obtain efficient semantic representation. Then we employ span-attention to fuse the contextual information and sentence-level information to obtain the specific span representation. Finally, we use the full connection layer to classify the entities and relations. Without introducing other external complicated features, this model obtains rich semantics and takes full advantage of the association between entity and relation. The experimental results show that on the NYT10 and NYT11 datasets, the F1 of the proposed model in the relation extraction task reaches 70.6% and 68.3% respectively, which is much better than other models.
关键词
实体关系抽取 /
门控多层感知机 /
BERT /
span-attention
{{custom_keyword}} /
Key words
entity and relation extraction /
MLPs with gating /
BERT /
span-attention
{{custom_keyword}} /
{{custom_sec.title}}
{{custom_sec.title}}
{{custom_sec.content}}
参考文献
[1] LIU X, CHEN H, XIA W. Overview of named entity recognition[J]. Journal of Contemporary Educational Research, 2022, 6(5): 65-68.
[2] CHO M, HA J, PARK C, et al. Combinatorial feature embedding based on CNN and LSTM for biomedical named entity recognition[J]. Journal of Biomedical Informatics, 2020, 103: 103381.
[3] WEN H, ZHU X, ZHANG L, et al. A gated piecewise CNN with entity-aware enhancement for distantly supervised relation extraction[J]. Information Processing & Management, 2020, 57(6): 102373.
[4] JIN Y, WU D, GUO W. Attention-based lstm with filter mechanism for entity relation classification[J]. Symmetry, 2020, 12, 1729.
[5] VERGA P, STRUBELL E, MCCALLUM A. Simultaneously self-attending to all mentions for full-abstract biological relation extraction[C]//Proceedings of the Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, 2018: 872-884.
[6] NGUYEN D Q, VERSPOOR K. End-to-end neural relation extraction using deep biaffine attention[C]//Proceedings of the European Conference on Information Retrieval. Springer, Cham, 2019: 729-738.
[7] YAO L, HUANG H, WANG K W, et al. Fine-grained mechanical Chinese named entity recognition based on ALBERT-AttBiLSTM-CRF and transfer learning[J]. Symmetry, 2020, 12(12), 1986.
[8] JIN G, YU Z. A Korean named entity recognition method using Bi-LSTM-CRF and masked self-attention[J]. Computer Speech & Language, 2021, 65, 101134.
[9] LI X, YIN F, SUN Z, et al. Entity-relation extraction as multi-turn question answering[C]//Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, 2019: 1340-1350.
[10] VU N T, ADEL H, GUPTA P, et al. Combining recurrent and convolutional neural networks for relation classification[C]//Proceedings of the Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, 2016: 534-539.
[11] LI, Q, JI H. Incremental joint extraction of entity mentions and relations[C]//Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics Baltimore, USA.
[12] MIWA M, SASAKI Y. Modeling joint entity and relation extraction with table representation[C] //Proceedings of the Conference on Empirical Methods in Natural Language Processing, 2014: 1858-1869.
[13] DIXIT K, AL-ONAIZAN Y. Span-level model for relation extraction[C]//Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, 2019: 5308-5314.
[14] LUAN Y, HE L, OSTENDORF M, et al. Multi-task identification of entities, relations, and corefer-ence for scientific knowledge graph construction[C]//Proceedings of the Conference on Empirical Methods in Natural Language Processing, 2018: 3219-3232.
[15] LUAN Y, WADDEN D, HE L, et al. A general framework for information extraction using dynamic span graphs[C]//Proceedings of NAACL-HLT, 2019: 3036-3046.
[16] EBERTS M, ULGES A. Span-based joint entity and relation extraction with transformer pre-training[M]. ECAI. IOS Press, 2020: 2006-2013.
[17] JI B, YU J, LI S, et al. Span-based joint entity and relation extraction with attention-based span-specific and contextual semantic representations[C]//Proceedings of the 28th International Conference on Computational Linguistics, 2020: 88-99.
[18] LI F, LIN Z C, ZHANG M, et al. A span-based model for joint overlapped and discontinuous named entity recognition[C]//Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing, 2021: 4814-4828.
[19] FU J, HUANG X J, LIU P. SpanNER: Named entity re-/recognition as span prediction[C]//Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing, 2021: 7183-7195.
[20] ZENG X, ZENG D, HE S, et al. Extracting relational facts by an end-to-end neural model with copy mechanism[C]//Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics, 2018: 506-514.
[21] ZENG X, ZENG D, HE S, et al. Extracting relational facts by an end-to-end neural model with copy mechanism[C]//Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics, 2018: 506-514.
[22] TAKANOBU R, ZHANG T Y, LIU J X, et al. A hierarchical framework for relation extraction with reinforcement learning[C]//Proceedings of the 33rd AAAI Conference on Artificial Intelligence, 2019:7072-7079.
[23] REN X, WU Z Q, HE W Q, et al. CoType: Joint extraction of typed entities and relations with knowledge bases[C]//Proceedings of the 26th International Conference on World Wide Web, 2017:1015-1024.
[24] GENG Z, ZHANG Y, HAN Y. Joint entity and relation extraction model based on rich semantics[J]. Neuro Computing, 2021, 429: 132-140.
{{custom_fnGroup.title_cn}}
脚注
{{custom_fn.content}}
PDF(2252 KB)
