基于深度学习的命名实体识别综述

PDF(2271 KB)

中文信息学报 ›› 2021, Vol. 35 ›› Issue (9) : 30-45.

综述

基于深度学习的命名实体识别综述

邓依依,邬昌兴,魏永丰,万仲保,黄兆华

作者信息 +

A Survey on Named Entity Recognition Based on Deep Learning

DENG Yiyi, WU Changxing, WEI Yongfeng, WAN Zhongbao, HUANG Zhaohua

Author information +

History +

摘要

命名实体识别是自然语言处理的基础任务之一,目的是从非结构化的文本中识别出所需的实体及类型,其识别的结果可用于实体关系抽取、知识图谱构建等众多实际应用。近些年,随着深度学习在自然语言处理领域的广泛应用,各种基于深度学习的命名实体识别方法均取得了较好的效果,其性能全面超越传统的基于人工特征的方法。该文从三个方面介绍近期基于深度学习的命名实体识别方法: 第一,从输入层、编码层和解码层出发,介绍命名实体识别的一般框架;第二,分析汉语命名实体识别的特点,着重介绍各种融合字词信息的模型;第三,介绍低资源的命名实体识别,主要包括跨语言迁移方法、跨领域迁移方法、跨任务迁移方法和集成自动标注语料的方法等。最后,总结相关工作,并提出未来可能的研究方向。

Abstract

Named entity recognition (NER), as one of the basic tasks in natural language processing, aims to identify the required entities and their types in unstructured text. In recent years, various named entity recognition methods based on deep learning have achieved much better performance than that of traditional methods based on manual features. This paper summarizes recent named entity recognition methods from the following three aspects: 1) A general framework is introduced, which consists of an input layer, an encoding layer and a decoding layer. 2) After analyzing the characteristics of Chinese named entity recognition, this paper introduces Chinese NER models which incorporate both character-level and word-level information. 3) The methods for low-resource named entity recognition are described, including cross-lingual transfer methods, cross-domain transfer methods, cross-task transfer methods, and methods incorporating automatically labeled data. Finally, the conclusions and possible research directions are given.

导出引用

邓依依,邬昌兴,魏永丰,万仲保,黄兆华. 基于深度学习的命名实体识别综述. 中文信息学报. 2021, 35(9): 30-45

DENG Yiyi, WU Changxing, WEI Yongfeng, WAN Zhongbao, HUANG Zhaohua. A Survey on Named Entity Recognition Based on Deep Learning. Journal of Chinese Information Processing. 2021, 35(9): 30-45

参考文献

[1] Sundheim B. Named entity task definition (v2.1)[C]//Proceedings of Message Understanding Conference, 1995: 319-332.
[2] Sang E F T K, Meulder F D. Introduction to the CoNLL-2003 shared task: Language-independent named entity recognition[C]//Proceedings of NAACL, 2003: 142-147.
[3] Li X, Sun X, Meng Y, et al. Dice loss for data-imbalanced NLP tasks[C]//Proceedings of ACL, 2020: 465-476.
[4] Lin B Y, Xu F, Luo Z, et al. Multi-channel biLSTM-CRF model for emerging named entity recognition in social media[C]//Proceedings of the 3rd Workshop on Noisy User-generated Text, 2017: 160-165.
[5] Wang J, Shou L, Chen K, et al. Pyramid: A layered model for nested named entity recognition[C]//Proceedings of ACL, 2020: 5918-5928.
[6] Sohrab M G, Miwa M. Deep exhaustive model for nested named entity recognition[C]//Proceedings of EMNLP, 2018: 2843-2849.
[7] Li Y, Chiticariu L, Reiss F, et al. Domain adaptation of rule-based annotators for named-entity recognition tasks[C]//Proceedings of EMNLP, 2010: 1002-1012.
[8] Gayen V, Sarkar K. An HMM based named entity recognition system for Indian languages: the JU System at ICON 2013[J]. CoRR, 2014, abs/1405.7397.
[9] 朱颢东, 杨立志, 丁温雪, 等. 基于主题标签和CRF的中文微博命名实体识别[J]. 华中师范大学学报(自然科学版), 2018, 052(3): 316-321.
[10] Su J, Tan Z, Xiong D, et al. Lattice-based recurrent neural network encoders for neural machine translation[C]//Proceedings of AAAI, 2017: 3302-3308.
[11] Devlin J, Chang M-W, Lee K, et al. BERT: pre-training of deep bidirectional transformers for language understanding[C]//Proceedings of NAACL-HLT, 2019: 4171-4186.
[12] Cui L, Zhang Y. Hierarchically-refined label attention network for sequence labeling[C]//Proceedings of EMNLP-IJCNLP, 2019: 4115-4128.
[13] Wu C, Hu C, Li R, et al. Hierarchical multi-task learning with CRF for implicit discourse relation recognition[J]. Knowledge-Based Systems, 2020, 195: 105637.
[14] Yadav V, Bethard S. A survey on recent advances in named entity recognition from deep learning models[C]//Proceedings of COLING, 2018: 2145-2158.
[15] Li J, Sun A, Han J, et al. A survey on deep learning for named entity recognition[J]. IEEE Transactions on Knowledge and Data Engineering, 2020.
[16] Huang Z, Xu W, Yu K. Bidirectional LSTM-CRF models for sequence tagging[J]. arXiv preprint arXiv:1508.01991,2015.
[17] Lample G, Ballesteros M, Subramanian S, et al. Neural architectures for named entity recognition[C]//Proceedings of NAACL-HLT, 2016: 260-270.
[18] Luo G, Huang X, Lin C Y, et al. Joint entity recognition and disambiguation[C]//Proceedings of EMNLP, 2015: 879-888.
[19] Chiu J P C, Nichols E. Named entity recognition with bidirectional LSTM-CNNS[J]. Transactions of the Association for Computational Linguistics, 2016, 4: 357-370.
[20] Yadav V, Sharp R, Bethard S. Deep affix features improve neural named entity recognizers[C]//Proceedings of Lexical and Computational Semantics, 2018: 167-172.
[21] Lin Y, Liu L, Ji H, et al. Reliability-aware dynamic feature composition for name tagging[C]//Proceedings of ACL, 2019: 165-174.
[22] Pradhan S, Moschitti A, Xue N, et al. Towards robust linguistic analysis using ontonotes[C]//Proceedings of CoNLL, 2013: 143-152.
[23] Vaswani A, Shazeer N, Parmar N, et al. Attention is all you need[C]//Proceedings of NIPS, 2017: 6000-6010.
[24] Strubell E, Verga P, Belanger D, et al. Fast and accurate entity recognition with iterated dilated convolutions[C]//Proceedings of EMNLP, 2017: 2670-2680.
[25] Chen H, Lin Z, Ding G, et al. GRN: gated relation network to enhance convolutional neural network for named entity recognition[C]//Proceedings of AAAI, 2019:6236-6243.
[26] Li P-H, Dong R-P, Wang Y-S, et al. Leveraging linguistic structures for named entity recognition with bidirectional recursive neural networks[C]//Proceedings of EMNLP, 2017: 2664-2669.
[27] Jie Z, Lu W. Dependency-guided LSTM-CRF for named entity recognition[C]//Proceedings of EMNLP-IJCNLP, 2019: 3862-3872.
[28] Guo Q, Qiu X, Liu P, et al. Star-transformer[C]//Proceedings of NAACL-HLT, 2019: 1315-1325.
[29] Yan H, Deng B, Li X, et al. TENER: Adapting transformer encoder for named entity recognition[J]. CoRR, 2019, abs/1911.04474.
[30] Shen Y, Yun H, Lipton Z C, et al. Deep active learning for named entity recognition[C]//Proceedings of the 2nd Workshop on Representation Learning for NLP, 2018: 252-256.
[31] Zhai F, Potdar S, Xiang B, et al. Neural models for sequence chunking[C]//Proceedings of AAAI, 2017: 3365-3371.
[32] Vinyals O, Fortunato M, Jaitly N. Pointer networks[C]//Proceedings of Advances in Neural Information Processing Systems 28, 2015: 2692-2700.
[33] Hinton G, Vinyals O, Dean J. Distilling the knowledge in a neural network[C]//Proceedings of NIPS Workshop. 2015.
[34] Liu W, Zhou P, Wang Z, et al. FastBERT: A self-distilling BERT with adaptive inference time[C]//Proceedings of ACL, 2020: 6035-6044.
[35] Sun Z, Yu H, Song X, et al. MobileBERT: A compact task-agnostic BERT for resource-limited devices[C]//Proceedings of ACL, 2020: 2158-2170.
[36] 王超, 王峥. 基于改进分词标注集的中文微博命名实体识别方法[J]. 计算机与数字工程, 2019, 47(1): 211-215.
[37] Tian Y, Song Y, Xia F, et al. Improving Chinese word segmentation with wordhood memory networks[C]//Proceedings of ACL, 2020: 8274-8285.
[38] Lu Y, Zhang Y, Ji D. Multi-prototype Chinese character embedding[C]//Proceedings of LREC, 2016: 855-859.
[39] Dong C, Zhang J, Zong C, et al. Character-based LSTM-CRF with radical-level features for Chinese named entity recognition[G]//Natural Language Understanding and Intelligent Applications, 2016, 10102: 239-250.
[40] Ma R, Peng M, Zhang Q, et al. Simplify the usage of lexicon in Chinese NER[C]//Proceedings of ACL, 2020: 5951-5960.
[41] 张海楠, 伍大勇, 刘悦, 等. 基于深度神经网络的中文命名实体识别[J]. 中文信息学报, 2017(4): 33-40.
[42] 殷章志, 李玖一. 融合字词模型的中文命名实体识别研究[J]. 中文信息学报, 2019, 33(11): 95-100.
[43] 石春丹, 秦岭. 基于BGRU-CRF的中文命名实体识别方法[J]. 计算机科学, 2019, 046(009): 237- 242.
[44] Peng N, Dredze M. Improving named entity recognition for Chinese social media with word segmentation representation learning[C]//Proceedings of ACL, 2016: 149-155.
[45] Cao P, Chen Y, Liu K, et al. Adversarial transfer learning for Chinese named entity recognition with self-attention mechanism[C]//Proceedings of EMNLP, 2018: 182-192.
[46] Wu F, Liu J, Wu C, et al. Neural Chinese named entity recognition via CNN-LSTM-CRF and joint training with word segmentation[C]//Proceedings of the World Wide Web Conference on - WWW, 2019: 3342-3348.
[47] Zhang Y, Yang J. Chinese NER using lattice LSTM[C]//Proceedings of ACL, 2018: 1554-1564.
[48] Gui T, Ma R, Zhang Q, et al. CNN-based Chinese NER with lexicon rethinking[C]//Proceedings of IJCAI, 2019: 4982-4988.
[49] Li X, Jie Z, Feng J, et al. Learning with rethinking: recurrently improving convolutional neural networks through feedback[J]. Pattern Recognition, 2018, 79: 183-194.
[50] Kim Y. Convolutional neural networks for sentence classification[C]//Proceedings of EMNLP, 2014: 1746-1751.
[51] Gui T, Zou Y, Zhang Q, et al. A lexicon-based graph neural network for Chinese NER[C]//Proceedings of EMNLP-IJCNLP, 2019: 1040-1050.
[52] Gilmer J, Schoenholz S S, Riley P F, et al. Neural message passing for quantum chemistry[C]//Proceedings of the 34th International Conference on Machine Learning, 2017: 2053-2070.
[53] Weischedel R, Pradhan S, Ramshaw L, et al. OntoNotes release 4.0. LDC2011T03, Philadelphia, Penn: Linguistic Data Consortium, 2011.
[54] Levow G-A. The third international Chinese language processing bakeoff: word segmentation and named entity recognition[C]//Proceedings of the 5th SIG-HAN Workshop on Chinese Language Processing, 2006: 108-117.
[55] Peng N, Dredze M. Named entity recognition for Chinese social media with jointly trained embeddings[C]//Proceedings of EMNLP, 2015: 548-554.
[56] Sui D, Chen Y, Liu K, et al. Leverage lexical knowledge for Chinese named entity recognition via collaborative graph network[C]//Proceedings of EMNLP-IJCNLP, 2019: 3828-3838.
[57] Liu W, Xu T, Xu Q, et al. An encoding strategy based word-character LSTM for Chinese NER[C]//Proceedings of NAACL-HLT, 2019: 2379-2389.
[58] Li X, Yan H, Qiu X, et al. FLAT: Chinese NER using flat-lattice transformer[C]//Proceedings of ACL, 2020: 6836-6842.
[59] Duan H, Zheng Y. A study on features of the CRFs-based Chinese named entity recognition[J]. International Journal of Advanced Intelligence Paradigms, 2011.
[60] Ju M, Miwa M, Ananiadou S. A neural layered model for nested named entity recognition[C]//Proceedings of NAACL-HLT, 2018: 1446-1459.
[61] Fisher J, Vlachos A. Merge and label: A novel neural network architecture for nested NER[C]//Proceedings of ACL, 2019: 5840-5850.
[62] Luo Y, Zhao H. Bipartite flat-graph network for nested named entity recognition[C]//Proceedings of ACL, 2020: 6408-6418.
[63] 李雁群, 何云琪, 钱龙华, 等. 中文嵌套命名实体识别语料库的构建[J]. 中文信息学报, 2018, 32(8): 19-26.
[64] Ni J, Dinu G, Florian R. Weakly supervised cross-lingual named entity recognition via effective annotation and representation projection[C]//Proceedings of ACL, 2017: 1470-1480.
[65] Mayhew S, Tsai C-T, Roth D. Cheap translation for cross-lingual named entity recognition[C]//Proceedings of EMNLP, 2017: 2536-2545.
[66] Koehn P, Zens R, Dyer C, et al. Moses: Open source toolkit for statistical machine translation[C]//Proceedings of ACL, 2007: 177-180.
[67] Xie J, Yang Z, Neubig G, et al. Neural cross-lingual named entity recognition with minimal resources[C]//Proceedings of EMNLP, 2018: 369-379.
[68] Wu S, Dredze M. Beto, Bentz, Becas: The surprising cross-lingual effectiveness of BERT[C]//Proceedings of EMNLP-IJCNLP, 2019: 833-844.
[69] Keung P, Lu Y, Bhardwaj V. Adversarial learning with contextual embeddings for zero-resource cross-lingual classification and NER[C]//Proceedings of EMNLP-IJCNLP, 2019: 1355-1360.
[70] Goodfellow I J, Pouget-Abadie J, Mirza M, et al. Generative adversarial networks[C]//Proceedings of NIPS, 2014: 2672-2680.
[71] Chen X, Awadallah A H, Hassan H, et al. Multi-source cross-lingual model transfer: Learning what to share[C]//Proceedings of ACL, 2019: 3098-3112.
[72] Bari M S, Joty S, Jwalapuram P. Zero-resource cross-lingual named entity recognition[C]//Proceedings of AAAI, 2020, 34: 7415-7423.
[73] Wu Q, Lin Z, Wang G, et al. Enhanced meta-Learning for cross-lingual named entity recognition with minimal resources[C]//Proceedings of AAAI. 2020, 34: 9274-9281.
[74] Wu Q, Lin Z, Karlsson B F, et al. Unitrans: Unifying model transfer and data transfer for cross-lingual named entity recognition with unlabeled data[C]//Proceedings of IJCAI, 2020: 3926-3932.
[75] Wu Q, Lin Z, Karlsson B F, et al. Single-/multi-source cross-lingual NER via teacher-student learning on unlabeled data in target language[C]//Proceedings of ACL, 2020: 6505-6514.
[76] Yang Z, Salakhutdinov R, Cohen W W. Transfer learning for sequence tagging with hierarchical recurrent Networks[C]//Proceedings of ICLR, 2017.
[77] Lin Y, Yang S, Stoyanov V, et al. A multi-lingual multi-task architecture for low-resource sequence labeling[C]//Proceedings of ACL, 2018: 799-809.
[78] Zhou J T, Zhang H, Jin D, et al. Dual adversarial neural transfer for low-resource named entity recognition[C]//Proceedings of ACL, 2019: 3461-3471.
[79] Jia C, Liang X, Zhang Y. Cross-domain NER using cross-domain language modeling[C]//Proceedings of ACL, 2019: 2464-2474.
[80] Liu Z, Winata G I, Fung P. Zero-resource cross-domain named entity recognition[C]//Proceedings of the 5th Workshop on Representation Learning for NLP, 2020: 1-6.
[81] He H, Sun X. A unified model for cross-domain and semi-supervised named entity recognition in chinese social media[C]//Proceedings of AAAI, 2017: 3216-3222.
[82] Lee J Y, Dernoncourt F, Szolovits P. Transfer learning for named-entity recognition with neural networks[C]//Proceedings of LREC, 2018: 4470-4473.
[83] Wang Z, Qu Y, Chen L, et al. Label-aware double transfer learning for cross-specialty medical named entity recognition[C]//Proceedings of NAACL, 2018: 1-15.
[84] Lin B Y, Lu W. Neural adaptation layers for cross-domain named entity recognition[C]//Proceedings of EMNLP, 2018:2012-2022.
[85] Wang J, Kulkarni M, Preotiuc-Pietro D. Multi-domain named entity recognition with genre-aware and agnostic inference[C]//Proceedings of ACL, 2020: 8476-8488.
[86] Collobert R, Weston J. A unified architecture for natural language processing: Deep neural networks with multi-task learning[C]//Proceedings of Machine Learning, 2008: 160.
[87] Sanh V, Wolf T, Ruder S. A hierarchical multi-task approach for learning embeddings from semantic tasks[C]//Proceedings of AAAI,2019, 33: 6949-6956.
[88] Aguilar G, Maharjan S, LPEZ Monroy A P, et al. A multi-task approach for named entity recognition in social media data[C]//Proceedings of the 3rd Workshop on Noisy User-generated Text, 2017: 148-153.
[89] Kruengkrai C, Nguyen T H, Aljunied S M, et al. Improving low-resource named entity recognition using joint sentence and token labeling[C]//Proceedings of ACL, 2020: 5898-5905.
[90] Rei M. Semi-supervised multitask learning for sequence labeling[C]//Proceedings of ACL, 2017: 2121-2130.
[91] Liu L, Shang J, Xu F F, et al. Empower sequence labeling with task-aware neural language model[C]//Proceedings of AAAI, 2018: 5253-5260.
[92] Cao Y, Hu Z, Chua T, et al. Low-resource name tagging learned with weakly labeled data[C]//Proceedings of EMNLP-IJCNLP, 2019: 261-270.
[93] Yang Y, Chen W, Li Z, et al. Distantly supervised ner with partial annotation learning and reinforcement learning[C]//Proceedings of Computational Linguistics, 2018: 2159-2169.
[94] Tsuboi Y, Kashima H, Oda H, et al. Training conditional random fields using incomplete annotations[C]//Proceedings of COLING, 2008, 1: 897-904.
[95] Feng J, Huang M, Zhao L, et al. Reinforcement learning for relation classification from noisy data[C]//Proceedings of AAAI, 2018: 5779-5786.
[96] Shang J, Liu L, Gu X, et al. Learning named entity tagger using domain-specific dictionary[C]//Proceedings of EMNLP, 2018: 2054-2064.
[97] Mayhew S, Chaturvedi S, Tsai C-T, et al. Named entity recognition with partially annotated training data[C]//Proceedings of CoNLL, 2019: 645-655.
[98] Peng M, Xing X, Zhang Q, et al. Distantly supervised named entity recognition using positive-unlabeled learning[C]//Proceedings of ACL, 2019: 2409-2419.
[99] Liang C, Yu Y, Jiang H, et al. BOND: BERT-assisted open-domain named entity recognition with distant supervision[C]//Proceedings of SIGKDD. 2020: 1054-1064.
[100] Lison P, Barnes J, Hubin A, et al. Named entity recognition without labelled data: A weak supervision approach[C]//Proceedings of ACL, 2020: 1518-1533.
[101] Safranchik E, Luo S, Bach S. Weakly supervised sequence tagging from noisy rules[C]//Proceedings of AAAI, 2020, 34: 5570-5578.
[102] Gui T, Ye J, Zhang Q, et al. Leveraging document-level label consistency for named entity recognition[C]//Proceedings of IJCAI, 2020: 3976-3982.

基金

国家重点研发计划(2018YFC0831106);国家自然科学基金(61866012);江西省自然科学基金(20181BAB202012);江西省教育厅科学技术研究项目(GJJ180329)

PDF(2271 KB)

4930

Accesses

Citation

Detail

段落导航

摘要
Abstract
关键词
Key words
引用本文
参考文献
基金

Received	Published
2020-08-24	2021-09-30
Issue Date
2021-09-30

选择文件类型/文献管理软件名称

选择包含的内容

摘要

Abstract

关键词

Key words

引用本文

{{custom_sec.title}}

{{custom_sec.title}}

参考文献

{{custom_fnGroup.title_cn}}

脚注

基金