关系分类是自然语言处理领域中重要的语义处理任务,随着机器学习技术的发展,预训练模型BERT在多项自然语言处理任务中取得了大量研究成果,但在关系分类领域尚有待探索。该文针对关系分类的问题特点,提出一种基于实体与实体上下文信息增强BERT的关系分类方法(EC_BERT),该方法利用BERT获取句子特征表示向量,并结合两个目标实体以及实体上下文语句信息,送入简单神经网络进行关系分类。此外,该文还对BERT的改进模型RoBERTa、DistilBERT进行了实验,发现BERT对于关系分类能力更为突出。实验结果显示,该方法在SemEval-2010任务8数据集和KBP-37数据集上Macro-F1值最高取得了89.69%和65.92%的结果,与以往方法相比,其在关系分类任务上表现出较好的性能。
Abstract
Relation classification is an important semantic processing task in the field of natural language processing. With the development of machine learning technology, the pre-trained model BERT has achieved excellent results in many natural language processing tasks. This paper proposes a relation classification method (EC_BERT) based on entity and entity context information enhanced BERT. This method uses BERT to obtain the sentence feature representation vector, and then combines two target entities and entity context statement information. In addition, the article also carried out experiments on RoBERTa model and DistiBERT model which are the improved model of BERT. The results on the SemEval-2010 task 8 dataset and the KBP-37 dataset show that the Bert based method performs best, achieving 89.69% and 65.92% of the Macro-F1, respectively.
关键词
关系分类 /
BERT /
自然语言处理 /
神经网络
{{custom_keyword}} /
Key words
relation classification /
BERT /
natural language processing /
neural network
{{custom_keyword}} /
{{custom_sec.title}}
{{custom_sec.title}}
{{custom_sec.content}}
参考文献
[1] Wang L, Cao Z, De Melo G, et al. Relation classification via multi-level attention CNNS[C]//Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics, 2016: 1298-1307.
[2] 刘峰,高赛,于碧辉,等.基于Multi-head attention和Bi-LSTM的实体关系分类[J].计算机系统应用,2019,28(06): 118-124.
[3] Liu C Y, Sun W B, Chao W H, et al. Convolution neural network for relation extraction[C]//Proceedings of the International Conference on Advanced Data Mining and Applications. Springer, Berlin, Heidelberg, 2013: 231-242.
[4] Zeng D, Liu K, Lai S, et al. Relation classification via convolutional deep neural network[C]//Proceedings of COLING 2014, the 25th International Conference on Conputeional Linguistics: Technical Paper. 2014: 2335-2344.
[5] Santos C N, Xiang B, Zhou B. Classifying relations by ranking with convolutional neural networks[J]. arXiv preprint arXiv: 1504.06580, 2015.
[6] Zhang D, Wang D. Relation classification via recurrent neural network[J]. arXiv preprint arXiv: 1508.01006, 2015.
[7] Socher R, Huval B, Manning C D, et al. Semantic compositionality through recursive matrix-vector spaces[C]//Proceedings of the 2012 Joint Conference on Empirical Methods in Natural Language Processing and Computational Natural Language Learning. Association for Computational Linguistics, 2012: 1201-1211.
[8] 宋睿,陈鑫,洪宇,等.基于卷积循环神经网络的关系抽取[J].中文信息学报,2019,33(10): 64-72.
[9] Zhou P, Shi W, Tian J, et al. Attention-based bidirectional long short-term memory networks for relation classification[C]//Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics, 2016: 207-212.
[10] Zhang L, Xiang F. Relation classification via BiLSTM-CNN[C]//Proceedings of the International Conference on Data Mining and Big Data. Springer, Cham, 2018: 373-382.
[11] Guo X, Zhang H, Yang H, et al. A single attention-based combination of CNN and RNN for relation classification[J]. IEEE Access, 2019, 7: 12467-12475.
[12] Radford A, Narasimhan K, Salimans T, et al. Improving language understanding by generative pre-training[J/OL]. https://s3-us-west-2. amazonaws.com/openai-assets/researchcovers/languageunsupervised/language understanding paper. pdf, 2018.
[13] Devlin J, Chang M W, Lee K, et al. BERT: Pre-training of deep bidirectional transformers for language understanding[J]. arXiv preprint arXiv: 1810.04805, 2018.
[14] Liu Y, Ott M, Goyal N, et al. RoBERTa: A robustly optimized BERT pretraining approach[J]. arXiv preprint arXiv: 1907.11692, 2019.
[15] Sanh V, Debut L, Chaumond J, et al. DistiBERT: A distilled version of BERT: smaller, faster, cheaper and lighter[J]. arXiv preprint arXiv: 1910.01108, 2019.
[16] Shi P, Lin J. Simple BERT models for relation extraction and semantic role labeling[J]. arXiv preprint arXiv: 1904.05255, 2019.
[17] Wu S, He Y. Enriching pre-trained language model with entity information for relation classification[J]. arXiv preprint arXiv: 1905.08284, 2019.
[18] Vaswani A, Shazeer N, Parmar N, et al. Attention is all you need[C]//Proceedings of the 31st International Conference on Neural Information Processing Systems, 2017: 6000-6010.
[19] 李俊,吕学强.融合BERT语义加权与网络图的关键词抽取方法[J].计算机工程: 2020, 46(9): 89-94.
{{custom_fnGroup.title_cn}}
脚注
{{custom_fn.content}}
基金
国家重点研发计划(2018YFC0807806);天诚汇智基金(2018A01003);北京联合大学研究生资助项目
{{custom_fund}}
PDF(2527 KB)
