融合预训练语言模型和标签依赖知识的关系抽取方法

Abstract
Figure/Table
References
Related Citation (15)

Download: PDF (3302 KB) HTML (1 KB)
Export: BibTeX | EndNote (RIS)

Abstract Relation extraction aims to extract the relations between entities from unlabeled free text. This paper proposes a relation extraction model that combines the pre-trained language model and label dependency knowledge. Specifically, given a sentence as the input, we first generate a deep contextualized word representation for the sentence and the two target entities using a pre-trained BERT encoder. At the same time, a multi-layer graph convolutional network is applied to model the dependency graph between the relation labels. Finally, we combine the above information to guide the relation classification. The experimental results show that our approach significantly outperforms the baselines.

Key words： relation extraction pre-trained language model label dependencies graph convolutional networks

Received: 29 January 2021

ZTFLH:

TP391

	Service

	E-mail this article
	Add to my bookshelf
	Add to citation manager
	E-mail Alert
	RSS
	Articles by authors
	ZHAO Chao
	XIE Songxian
	ZENG Daojian
	ZHENG Fei
	CHENG Chen
	PENG Lihong

Cite this article:

ZHAO Chao,XIE Songxian,ZENG Daojian, et al. Combination of Pre-trained Language Model and Label Dependency for Relation Extraction[J]. , 2022, 36(1): 75-82.

URL:

http://jcip.cipsc.org.cn/EN/ OR http://jcip.cipsc.org.cn/EN/Y2022/V36/I1/75

[1] Ye H, Chao W, Luo Z, et al. Jointly extracting relations with class ties via effective deep ranking[J]. arXiv preprint arXiv:1612.07602, 2016.
[2] Riedel S, Yao L, McCallum A. Modeling relations and their mentions without labeled text[C]//Proceedings of Joint European Conference on Machine Learning and Knowledge Discovery in Databases, 2010: 148-163.
[3] Kipf T N, Welling M. Semi-supervised classification with graph convolutional networks[J]. arXiv preprint arXiv:1609.02907, 2016.
[4] Devli J, Chang M W, Lee K, et al. BERT: Pre-training of deep bidirectional transformers for language understanding[C]//Proceedings of the Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, 2019,(1): 4171-4186.
[5] Zeng D, Liu K, Lai S,et al.Relation classification via convolutional deep neural network[C]//Proceedings of the 25th International Conference on Computational Linguistics: Technical Papers,2014: 2335-2344.
[6] Nguyen T H, Grishman R. Combining neural networks and log-linear models to improve relation extraction[J]. arXiv preprint arXiv:1511.05926, 2015.
[7] Vu N T, Adel H, Gupta P, et al. Combining recurrent and convolutional neural networks for relation classification[C]// Proceedings of the Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, 2016: 534-539.
[8] Lee J, Seo S, Choi Y S. Semantic relation classification via bidirectional LSTM networks with entity-aware attention using latent entity typing[J]. Symmetry, 2019, 11(6): 785.
[9] Cai R, Zhang X, Wang H. Bidirectional recurrent convolutional neural network for relation classification[C]//Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics, 2016(1): 756-765.
[10] Zhang Y, Qi P, Manning C D. Graph convolution over pruned dependency trees improves relation extraction[J]. arXiv preprint arXiv:1809.10185, 2018.
[11] Zhu H, Lin Y, Liu Z, et al. Graphneural networks with generated parameters for relation extraction[C]//Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, 2019: 1331-1339.
[12] Fu T J, Li P H, Ma W Y. GraphRel: Modeling text as relational graphs for joint entity and relation extraction[C]//Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics,2019: 1409-1418.
[13] Luan Y, Wadden D, He L, et al. A general framework for information extraction using dynamic span graphs[C]//Proceedings of the Conference of the North American Chapter of the Association for Computational Linguistics, 2019: 3036-3046.
[14] Christopoulou F, Miwa M, Ananiadou S. Awalk-based model on entity graphs for relation extraction[C]//Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics, 2018: 81-88.
[15] Alt C, Hübner M, Hennig L. Improving relation extraction by pre-trained language representations[J]. arXiv preprint arXiv:1906.03088, 2019.
[16] Wu S, He Y. Enriching pre-trained language model with entity information for relation classification[C]//Proceedings of the 28th ACM International Conference on Information and Knowledge Management, 2019: 2361-2364.
[17] Soares LB, Fitz Gerald N, Ling J. et al. Matching the blanks: Distributional similarity for relation learning[C]//Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, 2019: 2895-2905.
[18] Chen Z M, Wei X S, Wang P,et al.Multi-label image recognition with graph convolutional networks [C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2019: 5177-5186.
[19] Zeng X, Zeng D, He S,et al. Extracting relational facts by an end-to-end neural model with copy mechanism[C]//Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics, 2018,(1): 506-514.
[20] Li Qimai, Zhichao Han,Xiaoming Wu. Deeper insights into graph convolutional networks for semi-supervised learning[C]//Proceedings of the AAAI Conference on Artificial Intelligence, 2018: 3538-3545.