高质量的命名实体识别算法往往依赖海量的高质量标注数据来帮助实体识别模型的训练,然而大规模标注数据的获取存在诸多困难,如何通过文本信息自身的相关性来提高实体识别的准确性受到越来越多科研工作者的关注。该文有效地利用文本信息的语义相关性引入“实体联合器”概念,通过其与实体的高相关性,提高实体的数字化表征能力,进而实现对实体的有效识别。基于此,首先提出了一种实体联合器识别模型,通过文本关联结构信息来实现非标签文本的实体联合器识别;之后,采用经典的BiLSTM网络模型,提取句子的语义表征,并利用特征融合机制实现实体联合器与句子特征融合;由于实体联合器与实体有较强的关联性,又提出了针对实体表征及句子整体表征的约束机制,确保实体联合器在特征学习过程中的指导作用,精准高效地识别文本数据中的实体。通过在公开的数据集CoNLL03、NCBI Disease上对该文算法进行测试,相关实验结果证明了该文所提出算法的优越性和合理性。
Abstract
High-quality named entity recognition algorithms tend to rely on massive amounts of high-quality annotated data. However, there are many difficulties in obtaining large-scale annotated data. Therefore, more and more researchers pay attention to how to improve the accuracy of entity recognition through the relevance of text information. The concept of "entity combiner" is introduced to improve the entity's digital representation ability through its high relevance with entities. Then, the entity combiner recognition model is proposed to identify the entity combiner in the unlabeled text. The classical BILSTM(Bi-directional Long Short-Term Memory) network model is used to extract the semantic representation in sentences. Moreover, the feature-fused mechanism is implemented to combine the entity combiner and sentence feature. Due to the strong correlation between entity combiner and entity, the constraint mechanism for entity representation and sentence representation is proposed to ensure the function of entity combiner in the feature learning process. Experiments on CoNLL03 and NCBI Disease datasets demonstrate the superiority and effectiveness of the proposed method.
关键词
命名实体识别 /
语义相关性 /
实体联合器
{{custom_keyword}} /
Key words
named entity recognition /
semantic relevance /
entity combiner
{{custom_keyword}} /
{{custom_sec.title}}
{{custom_sec.title}}
{{custom_sec.content}}
参考文献
[1] LIN B Y, XU F, LUO Z, et al. Multichannel BiLSTM-CRF model for emerging named entity recognition in social media[C]//Proceedings of the 3rd Workshop on Noisy User-generated Text, 2017: 160-165.
[2] WANG X, YANG C, GUAN R. A comparative study for biomedical named entity recognition[J]. International Journal of Machine Learning and Cybernetics, 2015, 9(3): 373-382.
[3] LAMPLE G, BALLESTEROS M, SUBRAMANIAN S, et al. Neural architectures for named entity recognition[C]//Proceedings of the Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, 2016: 260-270.
[4] MA X, HOVY E. End-to-end sequence labeling via bi-directional LSTM-CNNs-CRF[J]. arXiv preprint arXiv: 1603.01354, 2016.
[5] CETOLI A, BRAGAGLIA S, SLOAN M, et al. Graph convolutional networks for named entity recognition[C]//Proceedings of the 16th International Workshop on Treebanks and Linguistic Theories, 2018: 37-45.
[6] HAMMERTON J. Named entity recognition with long short-term memory[C]//Proceedings of the 7th Annual Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, 2003: 172-175.
[7] COLLOBERT R, WESTON J, BOTTOU L, et al. Natural language processing (almost) from scratch[J]. Journal of Machine Learning Research, 2011, 12(8): 2493-2537.
[8] HUANG, Z, XU, W, YU, K. Bidirectional LSTM-CRF models for sequence tagging [J]. arXiv preprint arXiv: 1508.01991, 2015.
[9] BAHDANAU D, CHO K, BENGIO Y. Neural machine translation by jointly learning to align and translate[J]. arXiv preprint arXiv: 1409.0473, 2016.
[10] TAN Z, WANG M, XIE J, et al. Deep semantic role labeling with self-attention[C]//Proceedings of the American Association for Artificial Intelligence, 2017: 4923-4924.
[11] MATTHEW P, MARK N, MOHIT I, et al. Deep contextualized word representations[C]//Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics, 2018: 2227-2237.
[12] DEVLIN J, CHANG M W, LEE K, et al. BERT: Pre-training of deep bidirectional trans formers for language understanding[J].arXiv preprint arXiv: 1810.04805, 2018.
[13] SHANG J, LIU L, GU X, et al. Learning named entity tagger using domain-specific dictionary[C]//Proceedings of the Conference on Empirical Methods in Natural Language Processing, 2018: 2054-2064.
[14] YANG Y, CHEN W, LI Z, et al. Distantly supervised NER with partial annotation learning and reinforcement learning[C]//Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics, 2018: 2159-2169.
[15] LIN B Y, DONGHO L, FRANK F, et al. AlpacaTag: An active learning-based crowd annotation framework for sequence tagging[C]//Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics: System Demonstrations, 2019: 58-63.
[16] SHEN Y Y, HYOKUN Y, ZACHARY C, et al. Deep active learning for named entity recognition[C]//Proceedings of the 2nd Workshop on Representation Learning for NLP, 2017: 252-256.
[17] GUAN G, ZHU M. New research on transfer learning model of named entity recognition[J]. Journal of Physics Conference Series, 2019, 1267: 012-017.
[18] MEHMOOD T, GEREVINI A E, LAVELLI A, et al. Combining multi-task learning with transfer learning for biomedical named entity recognition[J]. Procedia Computer Science, 2020: 848-857.
[19] LAN O, HUANG X, LIN B Y, et al. Learning to contextually aggregate multi-source supervision for sequence labeling[C]//Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, 2020: 2134-2146.
[20] LIN B Y, LEE D H, SHEN M, et al. TriggerNER: Learning with entity triggers as explanations for named entity recognition[J]. arXiv preprint arXiv: 2004.07493, 2020.
[21] JEFFREY P, RICHARD S, CHRISTOPHER D M. Glove: Global vectors for word representation[C]//Proceedings of the Conference on Empirical Methods in Natural Language Processing, 2014: 1532-1543.
[22] XIN Y W, ETHAN H, VIBHUTI M, et al. Learning better internal structure of words for sequence labeling[C]//Proceedings of the Conference on Empirical Methods in Natural Language Processing, 2018: 2584-2593.
[23] REX Y, LOU Z, et al. Neural subgraph matching[J]. arXiv preprint arXiv: 2007.03092, 2020.
[24] LAFFERTY J, MCCALLUM A, FERNANDO P. Conditional random fields: Probabilistic models for segmenting and labeling sequence data[C]//Proceedings of the 18th International Conference on Machine Learning, 2001: 282-289.
[25] LIN Z, FENG M, SANTOS C N, et al. A structured self attentive sentence embedding[J]. arXiv preprint arXiv: 1703.03130, 2017.
[26] LUONG T, PHAM H, MANNING C D. Effective approaches to attention based neural machine translation[J]. arXiv preprint arXiv: 1508.04025, 2015.
[27] ERIK F, TJONG K S, FIEN D M. Introduction to the CoNLL shared task: Language independent named entity recognition[C]//Proceedings of the 7th Annual Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, 2003: 142-147.
[28] DOGAN R I, LEAMAN R, LU Z. NCBI disease corpus: A resource for disease name recognition and concept normalization[J]. Journal of Biomedical Informatics, 2014, 47: 1-10.
[29] YANG J, ZHANG Y. NCRF++: An opensource neural sequence labeling toolkit[C]//Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics, 2018, 74-79.
[30] QIAN Y J, ENRICO S, ZHIJING J, et al. GraphIE: A graph-based framework for information extraction[C]//Proceedings of the Annual Conference of the North American Chapter of the Association for Computational Linguistics, 2019: 751-761.
[31] REIMERS N, IRYNA G. Alternative weighting schemes for ELMo embeddings[J]. arXiv preprint arXiv: 1904.0295, 2019.
[32] SOUZA F, NOGUEIRA R, LOTUFO R. Portuguese named entity recognition using BERT-CRF[J]. arXiv preprint arXiv: 1909.10649, 2019.
[33] ZHAO Z, YANG Z, LUO L, et al. Disease named entity recognition from biomedical literature using a novel convolutional neural network[J]. Bmc Medical Genomics, 2017, 10(S5): 73.
[34] YOON W, SO C H, LEE J, et al. CollaboNet: Collaboration of deep neural networks for biomedical named entity recognition[J]. BMC Bioinform, 2019: 55-65.
[35] LING Y, HASAN S A, FARRI O, et al. A domain knowledge-enhanced LSTM-CRF model for disease named entity recognition[J]. AMIA Joint Summits on Translational Science Proceedings, 2019: 761-770.
[36] WANG X, Z, et al. Cross-type biomedical named entity recognition with deep multi-task learning[J]. Bioinform, 2019, 35(10): 1745-1752.
[37] DERBEL H, CHAIBI A H, GHEZALA H H B. Disease named entity recognition using long-short dependencies[J]. Journal of Bioinformatics and Computational Biology, 2020, 18(3): 1-15.
{{custom_fnGroup.title_cn}}
脚注
{{custom_fn.content}}
基金
国家重点研究与发展计划(2020YFB1711704);国家自然科学基金(62272337)
{{custom_fund}}
PDF(2400 KB)
