实体关系抽取任务是对句子中实体对间的语义关系进行识别。该文提出了一种基于Albert预训练语言模型结合图采样与聚合算法(Graph Sampling and Aggregation, GraphSAGE)的实体关系抽取方法,并在藏文实体关系抽取数据集上实验。该文针对藏文句子特征表示匮乏、传统藏文实体关系抽取模型准确率不高等问题,提出以下方案: ①使用预先训练的藏文Albert模型获得高质量的藏文句子动态词向量特征; ②使用提出的图结构数据构建与表示方法生成GraphSAGE模型的输入数据,并通过实验证明了该方法的有效性; ③借鉴GraphSAGE模型的优势,利用其图采样与聚合操作进行关系抽取。实验结果表明,该文方法有效提高了藏文实体关系抽取模型的准确率,且优于基线实验效果。
Abstract
Entity relation extraction task is to recognize relations between different entities in sentences. This paper proposes a Graph Sampling and Aggregation (GraphSAGE) Tibetan entity relation extraction method based on Albert pre-trained model. The language model is used to obtain high quality sentence features. Input data for the GraphSAGE model is generated by the graph structure data construction and representation method designed in this paper. The experimental results show that our method is effective and superior to the baseline methods.
关键词
藏文 /
实体关系抽取 /
Albert /
GraphSAGE
{{custom_keyword}} /
Key words
Tibetan /
entity relation extraction /
Albert /
GraphSAGE
{{custom_keyword}} /
{{custom_sec.title}}
{{custom_sec.title}}
{{custom_sec.content}}
参考文献
[1] Lan Z,Chen M,Goodman S,et al. ALBERT: A lite BERT for self-supervised learning of language representations[C]//Proceedings of International Conference on Learning Representations,2019: 1-17.
[2] William L H,Ying Z,Jure L. Inductive representation learning on large graphs[C]//Proceedings of International Conference on Neural Information Processing Systems,2017: 1024-1034.
[3] Hinton G E. Learning distributed representations of concepts[C]//Proceedings of the 8th Annual Conference of the Cognitive Science Society, 1989: 1-12.
[4] Bengio Y,Ducharme R,Vincent P,et al. A neural probabilistic language model[J]. Journal of Machine Learning Research,2003,3: 1137-1155.
[5] Mikolov T,Chen K,Corrado G,et al. Efficient estimation of word representations in vector space[J]. arXiv preprint arXiv: 1301.3781,2013.
[6] Joulin A,Grave E,Bojanowski P,et al. Bag of tricks for efficient text classification[C]//Proceedings of the 15th Conference of the European Chapter of the Association for Computational Linguistics,2017: 427-431.
[7] Vaswani A,Shazeer N,Parmar N,et al. Attention is all you need[C]//Proceedings of International Conference on Neural Information Processing Systems,2017: 5998-6008.
[8] Devlin J,Chang M W,Lee K,et al. BERT: Pretraining of deep bidirectional transformers for language understanding[J].arXiv preprint arXiv: 1810.04805,2018.
[9] Santos C,Bing X,Zhou B. Classifying relations by ranking with convolutional neural networks[J]. Computer science,2015,86(86): 132-137.
[10] Wang L,Zhu C,Melo G D,et al. Relation classification via multi-level attention CNNs[C]//Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics, 2016: 1298-1307.
[11] Chen G,Tian Y,Song Y,et al. Relation extraction with type-aware map memories of word dependencies[C]//Proceedings of the Association for Computational Linguistics,2021: 2501-2512.
[12] 夏天赐,孙媛. 基于联合模型的藏文实体关系抽取方法研究[J]. 中文信息学报,2018,32(12): 76-83.
[13] 郭莉莉,孙媛. 基于BP神经网络的藏语实体关系抽取[J/OL]. 软件导刊,2019,18(03): 7-9.
[14] 王丽客,孙媛,夏天赐. 基于远程监督的藏文实体关系抽取[J]. 中文信息学报,2020,34(03): 72-79.
[15] Bresson X,Vandergheynst P,Defferrard M. Convolutional neural networks on graphs with fast localized spectral filtering[C]//Proceedings of International Conference on Neural Information Processing Systems,2016: 3837-3845.
[16] Zaremba W,Sutskever I,Vinyals O. Recurrent neural network regularization[J]. arXiv preprint arXiv: 1409.2329,2014.
[17] Kim Y. Convolutional neural networks for sentence classification[C]//Proceedings of the Conference on Empirical Methods in Natural Language Processing,2014: 1746-1751.
[18] Johnson R,Tong Z. Deep pyramid convolutional neural networks for text categorization[C]//Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics,2017: 562-570.
[19] Lai S,Xu L,Liu K,et al. Recurrent convolutional neural networks for text classification[C]//Proceedings of the 29th AAAI Conference on Artificial Intelligence,2015: 2267-2273.
[20] Si L A,Yousfi A,Laaroussi S. A new estimate of the N-gram language model[J]. Procedia Computer Science,2021,189: 211-215.
{{custom_fnGroup.title_cn}}
脚注
{{custom_fn.content}}
基金
科技部重点研发计划重点专项(2017YFB1402200);西藏自治区科技创新基地自主研究项目(XZ2021JR002G)
{{custom_fund}}
PDF(6760 KB)
