短文本实体链接由于缺乏主题信息,只能依靠局部短文本信息和知识库。现有方法主要通过计算局部短文本和候选实体之间的相似度完成候选实体集的排序,但并未显式地考虑局部短文本和候选实体在文本交互上的关联性。针对上述问题,该文提出短文本交互图(STIG)的概念和一个双步训练方案,利用BERT提取局部短文本和候选实体间的多粒度特征,并在短文本交互图上使用图卷积机制。此外,为了缓解均值池化使图卷积发生退化的问题,该文提出一个将交互图中各节点特征和边信息压缩成稠密向量的方法。在CCKS2020短文本实体链接数据集上的实验验证了所提方法的有效性。
Abstract
Short text entity linking can relies on local short text information and knowledge base due to the lack of global topic information. This paper proposes the concept of short text interaction graph (STIG) and a double stage training strategy. The Bert is used to extract the multi-granularity features between local short text and candidate entities, and the graph convolution mechanism is used on the short text interaction graph. To alleviate the degradation of graph convolution caused by mean pooling, a method is further proposed to compress the feature of nodes and edges information in interaction graph into a dense vector. Experiments on CCKS2020 entity linking dataset show the effectiveness of the proposed method.
关键词
实体链接 /
BERT /
图卷积神经网络
{{custom_keyword}} /
Key words
entity linking /
BERT /
graph convolutional neural network
{{custom_keyword}} /
{{custom_sec.title}}
{{custom_sec.title}}
{{custom_sec.content}}
参考文献
[1] Cucerzan S. Large-Scale named entity disambiguation based on Wikipedia data[C]//Proceedings of the Joint Conference on Empirical Methods in Natural Language Processing and Computational Natural Language Learning,2007: 708-716.
[2] Globerson A, Strube M, Chakrabarti S, et al. Collective entity resolution with multi-focal attention[C]//Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics,2016: 621-631.
[3] Heinzerling B, Michael S, Lin C Y. Trust, but verify! better entity linking through automatic verification[C]//Proceedings of the 15th Conference of the European Chapter of the Association for Computational Linguistics,2017: 828-838.
[4] Devlin J, Chang M W, Lee K, et al. BERT: pre-training of deep bidirectional transformers for language understanding[J].arXiv preprint, arXiv: 1810.04805, 2018.
[5] Radford A,Narasimhan K, Salimans T, et al. Improving language understanding by generative pre-training[EB/OL]. https://cdn.openai.com/research-covers/language-unsupervised/language_understanding_paper.pdf.
[6] Kipf T N, Welling M. Semi-supervised classification with graph convolutional networks[J]. arXiv preprint, arXiv: 1609.02907, 2016.
[7] Zhang W, Su J, Tan C L, et al. Entity linking leveraging: automatically generated annotation[C]//Proceedings of the 23rd International Conference on Computational Linguistics. Association for Computational Linguistics,2010: 1290-1298.
[8] Raiman J, Raiman O. DeepType: Multilingual entity linking by neural type system evolution[C]//Proceedings of the 32rd Conference on Association for the Advancement of Artificial Intelligence,2018: 5406-5413.
[9] Ganea O E, Hofmann T. Deep joint entity disambiguation with local neural attention[C]//Proceedings of the Conference on Empirical Methods in Natural Language Processing,2017: 2619-2629.
[10] 李明扬,姜嘉伟,孔芳. 融入丰富信息的高性能神经实体链接[J]. 中文信息学报,2020,34(01): 87-96.
[11] 周鹏程,武川,陆伟. 基于多知识库的短文本实体链接方法研究: 以Wikipedia和Freebase为例[J]. 现代图书情报技术,2016, 32(06): 1-11.
[12] Nie F, Zhou S, Wang J, et al. Aggregated semantic matching for short text entity linking[C]//Proceedings of the 22nd Conference on Computational Natural Language Learning,2018: 476-485.
[13] Herbrich R, Minka T, Graepel T. TrueSkilltm: A bayesian skill rating system[C]//Proceedings of the 19rd Conference on Neural Information Processing Systems,2006: 569-576.
[14] Logeswaran L, Chang M W, Lee K, et al. Zero-shot entity linking by reading entity descriptions[C]//Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics,2019: 3449-3460.
[15] Chen S, Wang J, Jiang F, et al. Improving entity linking by modeling latent entity type information[J]. arXiv preprint, arXiv: 2001.01447, 2020.
[16] Vaswani A, Shazeer N, Parmar N, et al. Attention is all you need[C]//Proceedings of the 31th Advances in Neural Information Processing Systems,2017: 6000-6010.
[17] Cui Y, Che W, Liu T, et al. Revisiting pretrained models for Chinese natural language processing[J]. arXiv preprint, arXiv: 2004.13922,2004.
[18] Krizhevsky A, Sutskever I, Hinton G. Imagenet classification with deep convolutional neural networks[C]//Proceedings of the 26rd Neural Information Processing Systems,2012: 1097-1105.
[19] Baidu, CCKS. CCKS 2020: 面向中文短文本的实体链指任务[DB/OL].https://www.biendata.xyz/competition/ccks_2020_el/.
[20] Loshchilov I, Hutter F. Decoupled weight decay regularization[C]//Proceedings of the 7th International Conference on Learning Representations,2019, arXiv: 1711.05101.
[21] Kingma D P, Ba J. Adam: A method for stochastic optimization[C]//Proceedings of the 3th International Conference on Learning Representations,2015, arXiv: 1412.6980.
[22] Xue M, Cai W, Su J, et al. Neural collective entity linking based on recurrent random walk network learning[C]//Proceedings of the 28rd International Joint Conference on Artificial Intelligence,2019: 5327-5333.
[23] Yin X, Huang Y, Zhou B, et al. Deep entity linking via eliminating semantic ambiguity with BERT[J]. IEEE Access, 2019, 7: 169434-169445.
[24] Goodfellow I J, Shlens J, Szegedy C. Explaining and harnessing adversarial examples[J]. arXiv preprint arXiv: 1412.6572, 2014.
[25] LZhao Z, Chen H, Zhang J, et al. UER: An Open-Source Toolkit for Pre-training Models[J]. arXiv preprint arXiv: 1909.05658, 2019.
[26] Liu Y, Ott M, Goyal N, et al. Roberta: A robustly optimizedbert pretraining approach[J]. arXiv preprint arXiv: 1907.11692, 2019.
[27] 吕荣荣,王鹏程,陈帅. 面向中文短文本的多因子融合实体链指研究[EB/OL]. https://bj.bcebos.com/v1/conference/ccks2020/eval_paper/ccks2020_eval_paper_2_1.pdf.
{{custom_fnGroup.title_cn}}
脚注
{{custom_fn.content}}
基金
国家自然科学基金(U2003303);中国科学院西部青年学者项目-A类(2019-XBQNXZ-A-004);中国科学院青年创新促进会项目(科发人函字[2019]26号);自治区天山青年计划项目(2018Q032);新疆维吾尔自治区重大专项(2020A03004-4)
{{custom_fund}}
PDF(3453 KB)
