命名实体识别是文档级关系抽取中的一项关键任务,然而,传统的文档级关系抽取模型在实体识别时,仅通过汇聚局部提及信息构建实体,这限制了实体的表征能力。为此,该文提出了基于注意力机制补足实体缺陷的文档级关系抽取方法。该方法根据预定义的关系集合选择性关注实体提及层次特征,然后利用池化方法积累信号,为实体补足不同提及语义特征,同时,引入交叉多头注意力机制和残差连接对实体进行上下文加权处理,加强实体与上下文、全局信息之间的联系。该文在DocRED数据集上进行实验,与基线模型相比,补足实体缺陷后的基线模型在验证集F1/Ign_F1和测试集F1/Ign_F1上分别提升了1.82%/1.73%和1.81%/1.62%,实验结果表明了该方法的有效性。
Abstract
Named Entity Recognition (NER) is a vital task in document-level relation extraction. Traditional models in this domain construct entities by aggregating local mentions, thereby constraining entity representational capabilities. To address this limitation, this paper proposes a document-level relation extraction method that supplemented entity deficiencies through an attention mechanism. The approach concentrates on hierarchical features directed by predefined relations, and employs pooling to augment mention semantics. It introduces a cross-multi-head attention mechanism and residual connections for context-weighted processing, reinforcing associations among entities, context, and global information. Experiments on the DocRED dataset reveal improvements of 1.82%/1.73% and 1.81%/1.62% in the validation set F1/Ign_F1 and test set F1/Ign_F1, respectively.
关键词
文档级关系抽取 /
命名实体识别 /
注意力机制
{{custom_keyword}} /
Key words
document-level relation extraction /
named entity recognition /
attention mechanisms
{{custom_keyword}} /
{{custom_sec.title}}
{{custom_sec.title}}
{{custom_sec.content}}
参考文献
[1] 贾宝林,尹世群,王宁朝. 基于门控多层感知机的端到端实体关系联合抽取[J].中文信息学报,2023,37(3): 143-151.
[2] DENG Y, XIE Y, LI Y, et al. Multi-task learning with multi-view attention for answer selection and knowledge base question answering[C]//Proceedings of the 33rd AAAI Conference on Artificial Intelligence, 2019: 6318-6325.
[3] XU W, CHEN K, ZHAO T. Document-level relation extraction with reconstruction[C]//Proceedings of the AAAI Conference on Artificial Intelligence,2021:14167-14175.
[4] LIU J, SUI D, LIU K, et al. Graph-based knowledge integration for question answering over dialogue[C]//Proceedings of the 28th International Conference on Computational Linguistics, 2020: 2425-2435.
[5] 丁泽源,杨志豪,罗凌,等. 基于深度学习的中文生物医学实体关系抽取系统[J].中文信息学报,2021,35(5): 70-76.
[6] 李晓林,潘治霖,邓庆康,等. 基于融合关系信息编码的法律文书实体关系抽取方法[J].中文信息学报,2023,37(4): 90-97.
[7] 曹碧薇,曹玖新,桂杰,等. 面向中文文学作品的长文本人物关系抽取[J].中文信息学报,2023,37(5): 88-100.
[8] YE D, LIN Y, DU J, et al. Coreferential reasoning learning for language representation[C]//Proceedings of the Conference on Empirical Methods in Natural Language Processing, 2020: 7170-7186.
[9] LI J, XU K, LI F, et al. MRN: A locally and globally mention-based reasoning network for document-level relation extraction[C]//Proceedings of the Association for Computational Linguistics, 2021: 1359-1370.
[10] ZHOU W, HUANG K, MA T, et al. Document-level relation extraction with adaptive thresholding and localized context pooling[C]//Proceedings of the AAAI Conference on Artificial Intelligence, 2021: 14612-14620.
[11] XIE Y, SHEN J, LI S, et al. Eider: Empowering document-level relation extraction with efficient evidence extraction and inference-stage fusion[C]//Proceedings of the Association for Computational Linguistics, 2022: 257-268.
[12] CAI R, ZHANG X, WANG H, et al. Bidirectional recurrent convolutional neural network for relation classification[C]//Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics, 2016: 756-765.
[13] ZHANG Y, ZHONG V, CHEN D, et al. Position-aware attention and supervised data improve slot filling[C]//Proceedings of the Conference on Empirical Methods in Natural Language Processing, 2017: 35-45.
[14] SHI P, LIN J. Simplebert models for relation extraction and semantic role labeling[J]. arXiv preprint arXiv:1904.05255, 2019.
[15] YAO Y, YE D, LI P, et al.DocRED: A large-scale document-level relation extraction dataset[C]//Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, 2019: 764-777.
[16] CHENG Q, LIU J, QU X, et al. HacRED: A large-scale relation extraction dataset toward hard cases in practical applications[C]//Proceedings of the Association for Computational Linguistics, 2021: 2819-2831.
[17] QUIRK C, POON H. Distant supervision for relation extraction beyond the sentence boundary[C]//Proceedings of the 15th Conference of the European Chapter of the Association for Computational Linguistics, 2017: 1171-1182.
[18] PAN J, PENG M, ZHANG Y. Mention-centered graph neural network for document-level relation extraction[J].arXiv preprint arXiv:2103.08200, 2021.
[19] KIPF T N, WELLING M. Semi-supervised classification with graph convolutional networks[C]//Proceedings of the International Conference on Learning Representations, 2016.
[20] XU W, CHEN K, ZHAO T. Document-level relation extraction with reconstruction[C]//Proceedings of the AAAI Conference on Artificial Intelligence, 2021: 14167-14175.
[21] QI J, XU B, ZENG K, et al.ConstGCN: Constrained transmission-based graph convolutional networks for document-level relation extraction[J]. arXiv preprint arXiv:2210.03949, 2022.
[22] XU B, WANG Q, LYU Y, et al. Entity structure within and throughout: Modeling mention dependencies for document-level relation extraction[C]//Proceedings of the AAAI Conference on Artificial Intelligence, 2021: 14149-14157.
[23] YU J, YANG D, TIAN S. Relation-specific attentions over entity mentions for enhanced document-level relation extraction[C]//Proceedings of the Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, 2022: 1523-1529.
[24] JIA R, WONG C, POON H. Document-level N-ary relation extraction with multiscale representation learning[C]//Proceedings of the Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, 2019: 3693-3704.
[25] VASWANI A,SHAZEER N, PARMAR N, et al. Attention is all you need[C]//Proceedings of the Advances in Neural Information Processing Systems, 2017: 6000-6010.
[26] DEVLIN J, CHANG M, LEE K, et al. BERT: Pre-training of deep bidirectional transformers for language understanding[C]//Proceedings of the Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, 2019: 4171-4186.
[27] LIU Y, OTT M, GOYAL N, et al. RoBERTa: A robustly optimized bert pretraining approach[C]//Proceedings of the 25th Chinese National Conference on Computational Linguistics, 2021: 1218-1227.
[28] ZENG S, WU Y, CHANG B. SIRE: Separate intra- and inter-sentential reasoning for document-level relation extraction[C]//Proceedings of the Association for Computational Linguistics, 2021: 524-534.
[29] WEI Y, LI Q.SagDRE: Sequence-aware graph-based document-level relation extraction with adaptive margin loss[C]//Proceedings of the 28th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, 2022: 2000-2008.
[30] KUANG H, CHEN H, MA X, et al. A keyword detection and context filtering method for document level relation extraction[J]. Applied Sciences, 2022, 12(3): 1599-1611.
{{custom_fnGroup.title_cn}}
脚注
{{custom_fn.content}}
PDF(1367 KB)
