基于注意力与同指信息的对话级关系抽取

周孟佳,李霏,姬东鸿

PDF(3280 KB)
PDF(3280 KB)
中文信息学报 ›› 2024, Vol. 38 ›› Issue (1) : 97-106.
信息抽取与文本挖掘

基于注意力与同指信息的对话级关系抽取

  • 周孟佳,李霏,姬东鸿
作者信息 +

Dialogue-level Relation Extraction Based on Attention and Coreference

  • ZHOU Mengjia, LI Fei, JI Donghong
Author information +
History +

摘要

与传统的关系抽取任务相比,对话级关系抽取任务具有语言随意、信息密度低、人称代词丰富的特点。基于此,该文提出了一种基于注意力和同指信息的对话级关系抽取模型。模型采用TOD-BERT(Task-Oriented Dialogue BERT)和BERT预训练语言模型增强对话文本表示,通过注意力机制建模词与各种关系之间的相互影响,使模型能更多地关注有益信息。另外,该文提出的模型还融合了与人称代词相关的同指信息以丰富实体的表示。作者在对话级关系抽取数据集DialogRE上验证所提出模型的性能。结果表明,该模型在DialogRE测试集上的F1值达到了63.77%,较之于多个基线模型有明显提升。

Abstract

The dialog-level relation extraction is characterized by casual language, low information density and abundant personal pronouns. This paper proposes an end-to-end dialogue relation extraction model via TOD-BERT (Task-Oriented Dialogue BERT) pre-trained language model. It adopts the attention mechanism to capture the interaction between different words and different relations. Besides, the co-reference information related to personal pronouns is applied to enrich the entity features. Validated on DialogRE, a new dialog-level relational extraction dataset, the proposed model reaches 63.77 F1 score, which is significantly better than the baseline models.

关键词

关系抽取 / 注意力机制 / 同指信息 / 对话

Key words

relation extraction / attention mechanism / co-reference information / dialogue

引用本文

导出引用
周孟佳,李霏,姬东鸿. 基于注意力与同指信息的对话级关系抽取. 中文信息学报. 2024, 38(1): 97-106
ZHOU Mengjia, LI Fei, JI Donghong. Dialogue-level Relation Extraction Based on Attention and Coreference. Journal of Chinese Information Processing. 2024, 38(1): 97-106

参考文献

[1] CALì A, GOTTLOB G, PIERIS A. Query answering under expressive entity-relationship schemata[C]//Proceedings of the International Conference on Conceptual Modeling. Springer, Berlin, Heidelberg, 2010: 347-361.
[2] SHIN J, WU S, WANG F, et al. Incremental knowledge base construction using deepdive[C]//Proceedings of the VLDB Endowment International Conference on Very Large Data Bases. NIH Public Access, 2015, 8(11): 1310.
[3] HENDRICKX I, KIM S N, KOZAREVA Z, et al. SemEval-2010task 8: Multi-way classification of semantic relations between pairs of nominals[C]//Proceedings of the 5th International Workshop on Semantic Evaluation, 2010: 33-38.
[4] YAO Y, YE D, LI P, et al. DocRED: A large-scale document-level relation extraction dataset[C]//Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, 2019.
[5] CHRISTOPOULOU F, MIWA M, ANANIADOU S. Connecting the dots: Document-level neural relation extraction with edge-oriented graphs[C]//Proceedings of the Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing, 2019: 4927-4938.
[6] 庄传志,靳小龙,朱伟建,等.基于深度学习的关系抽取研究综述[J].中文信息学报,2019,33(12): 1-18.
[7] YU D, SUN K, CARDIE C, et al. Dialogue-based relation extraction[C]//Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, 2020: 4927-4940.
[8] DEVLIN J, CHANG M W, LEE K, et al. BERT: Pre-training of deep bidirectional transformers for language understanding[C]//Proceedings of the Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, 2019: 4171-4186.
[9] WU C S, HOI S C H, SOCHER R, et al.TOD-BERT: Pre-trained natural language understanding for task-oriented dialogue[C]//Proceedings of the Conference on Empirical Methods in Natural Language Processing, 2020: 917-929.
[10] DODDINGTON G R, MITCHELL A, PRZYBOCKI M A, et al. The automatic content extraction (ace) program-tasks, data, and evaluation[C]//Proceedings of the Lrec, 2004, 2(1): 837-840.
[11] WALKER C, STRASSEL S, MEDERO J, et al. ACE 2005 multilingual training corpus[J]. Linguistic Data Consortium, Philadelphia, 2006, 57: 45.
[12] ZENG D, LIU K, LAI S, et al. Relation classification via convolutional deep neural network[C]//Proceedings of COLING, the 25th International Conference on Computational Linguistics: Technical Papers, 2014: 2335-2344.
[13] MIWA M, BANSAL M. End-to-end relation extraction using LSTMs on sequences and tree structures[C]//Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics, 2016: 1105-1116.
[14] LI J, SUN Y, JOHNSON R J, et al. BioCreativeV CDR task corpus: A resource for chemical disease relation extraction[J]. Database, 2016,5: 9-20.
[15] WU Y, LUO R, LEUNG H C M, et al. Renet: A deep learning approach for extracting gene-disease associations from literature[C]//Proceedings of the International Conference on Research in Computational Molecular Biology. Springer, Cham, 2019: 272-284.
[16] PENG N, POON H, QUIRK C, et al. Cross-sentence n-ary relation extraction with graph lstms[J]. Transactions of the Association for Computational Linguistics, 2017, 5: 101-115.
[17] SAHU S K, CHRISTOPOULOU F, MIWA M, et al. Inter-sentence relation extraction with document-level graph convolutional neural network[C]//Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, 2019: 4309-4316.
[18] TANG H, CAO Y, ZHANG Z, et al. Hin: Hierarchical inference network for document-level relation extraction[C]//Proceedings of the Pacific-Asia Conference on Knowledge Discovery and Data Mining. Springer, Cham, 2020: 197-209.
[19] NAN G, GUO Z, SEKULIC I, et al. Reasoning with latent structure refinement for document-level relation extraction[C]//Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, 2020: 1546-1557.
[20] XUE F, SUN A, ZHANG H, et al.GDPNet: Refining latent multi-view graph for relation extraction[J/OL]. arXiv preprint arXiv: 2012.06780, 2020.
[21] ZHANG Y, ZHONG V, CHEN D, et al. Position-aware attention and supervised data improve slot filling[C]//Proceedings of the Conference on Empirical Methods in Natural Language Processing, 2017: 35-45.
[22] CHEN H, HONG P, HAN W, et al. Dialogue relation extractionwith document-level heterogeneous graph attention networks[J/OL]. arXiv preprint arXiv: 2009.05092, 2020.
[23] LI B, YE W, SHENG Z, et al. Graph enhanced dual attention network for document-level relation extraction[C]//Proceedings of the 28th International Conference on Computational Linguistics, 2020: 1551-1560.
[24] 周奇安, 李舟军. 基于BERT的任务导向对话系统自然语言理解的改进模型与调优方法[J]. 中文信息学报, 2020(5): 82-90.
[25] 李丹,徐童,郑毅,等.部首感知的中文医疗命名实体识别[J].中文信息学报,2020,34(12): 54-64.
[26] VASWANI A, SHAZEER N, PARMAR N, et al. Attention is all you need[C]//Proceedings of the Advances in Neural Information Processing Systems, 2017: 5998-6008.
[27] WOLF T. State-of-the-art neural coreference resolution for chatbots[EB/OL].https://medium.com/huygingface/state-of-the-art-heural-conference-resolution-for-chatbots-3302365dcf30[2020-08-11].
[28] KINGMA D P, BA J. Adam: A method for stochastic optimization[C]//Proceedings of the ICLR, 2015.
[29] WANG H, FOCKE C, SYLVESTER R, et al. Fine-tune bert for docred with two-step process[J/OL]. arXiv preprint arXiv: 1909.11898, 2019.
[30] QUAN J, ZHANG S, CAO Q, et al. RiSAWOZ: A large-scale multi-domain wizard-of-oz dataset with rich semantic annotations for task-oriented dialogue modeling[C]//Proceedings of the Conference on Empirical Methods in Natural Language Processing, 2020: 930-940.

基金

国家自然科学基金(61772378);国家重点研究与发展项目(2017YFC1200500);教育部研究基金(18JZD015)
PDF(3280 KB)

558

Accesses

0

Citation

Detail

段落导航
相关文章

/