张汝佳,代璐,王邦,郭鹏. 基于深度学习的中文命名实体识别最新研究进展综述[J]. 中文信息学报, 2022, 36(6): 20-35.
ZHANG Rujia, DAI Lu, WANG Bang, GUO Peng. Recent Advances of Chinese Named Entity Recognition Based on Deep Learning. , 2022, 36(6): 20-35.
基于深度学习的中文命名实体识别最新研究进展综述
张汝佳,代璐,王邦,郭鹏
华中科技大学 电子信息与通信学院,湖北 武汉 430074
Recent Advances of Chinese Named Entity Recognition Based on Deep Learning
ZHANG Rujia, DAI Lu, WANG Bang, GUO Peng
School of Electronic Information and Communications, Huazhong University of Science and Technology, Wuhan, Hubei 430074, China
Abstract：Chinese named entity recognition (CNER) is one of the basic tasks of natural language processing applications such as question answering systems, machine translation and information extraction. Although traditional CNER system has achieved satisfactory experiment results with the help of manually designed domain-specific features and grammatical rules, it is still defected in aspects such as weak generalization ability, poor robustness and difficult maintenance. In recent years, deep learning techniques have been adopted to deal with the above shortcomings by automatically extracting text features in an end-to-end learning manner. This article surveys the recent advances of deep learning-based CNER. It first introduces the concepts, difficulties and applications of CNER, and introduces the common datasets and evaluation metrics. Recent neural network models for the CNER task are then grouped according to their network architectures, and representative models in each group are detailed. Finally, future research directions are discussed.
[1] Xie R, Liu Z, Jia J, et al. Representation learning of knowledge graphs with entity descriptions[C]∥Proceedings of the AAAI Conference on Artificial Intelligence, 2016. [2] Guo J, Xu G, Cheng X, et al. Named entity recognition in query[C]//Proceedings of the 32nd international ACM SIGIR conference on Research and development in information retrieval, 2009: 267-274. [3] Petkova D, Croft W B. Proximity-based document representation for named entity retrieval[C]//Proceedings of the 16th ACM conference on Conference on information and knowledge management, 2007: 731-740. [4] Mollá D, Van Zaanen M, Smith D. Named entity recognition for question answering[C]//Proceedings of the Australasian Language Technology Workshop 2006, 2006: 51-58. [5] Babych B, Hartley A. Improving machine translation quality with automatic named entity recognition[C]//Proceedings of the 7th International EAMT workshop on MT and other language technology tools, Improving MT through other language technology tools, Resource and tools for building MT at EACL, 2003. [6] Sahu S K, Anand A. Recurrent neural network models for disease name recognition using domain invariant features[J]. arXiv preprint arXiv:1606.09371, 2016. [7] Zhao S, Liu T, Zhao S, et al. A neural multi-task learning framework to jointly model medical named entity recognition and normalization[C]//Proceedings of the AAAI Conference on Artificial Intelligence, 2019: 817-824. [8] Weischedel R, Pradhan S, Ramshaw L, et al. Ontonotes release 4.0[DB]. LDC2011T03, Philadelphia, Penn.: Linguistic Data Consortium, 2011. [9] Levow G A. The third international Chinese language processing bakeoff: Word segmentation and named entity recognition[C]//Proceedings of the 5th SIGHAN Workshop on Chinese Language Processing, 2006: 108-117. [10] Jin G, Chen X. The 4th international chinese language processing bakeoff: Chinese word segmentation, named entity recognition and chinese pos tagging[C]//Proceedings of the 6th SIGHAN Workshop on Chinese Language Processing, 2008. [11] Zhang Y, Yang J.Chinese ner using lattice lstm[J]. arXiv preprint arXiv:1805.02023, 2018. [12] Peng N,Dredze M. Named entity recognition for chinese social media with jointly trained embeddings[C]//Proceedings of the Conference on Empirical Methods in Natural Language Processing, 2015: 548-554. [13] He H, Sun X. F-score driven max margin neural network for named entity recognition inchinese social media[J]. arXiv preprint arXiv:1611.04234, 2016. [14] Ding R,Xie P, Zhang X, et al. A neural multi-digraph model for Chinese NER with gazetteers[C]//Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, 2019: 1462-1467. [15] Sang E F, DeMeulder F. Introduction to the CoNLL-2003 shared task: Language-independent named entity recognition[J]. arXiv preprint cs/0306050, 2003. [16] Pradhan S, Moschitti A, Xue N, et al. CoNLL-2012 shared task: Modeling multilingual unrestricted coreference in OntoNotes[C]//Proceedings of the Joint Conference on EMNLP and CoNLL-Shared Task, 2012: 1-40. [17] Lample G, Ballesteros M, Subramanian S, et al. Neural architectures for named entity recognition[J]. arXiv preprint arXiv:1603.01360, 2016. [18] Zhou J T, Zhang H,Jin D, et al. Roseq: Robust sequence labeling[J]. IEEE Transactions on Neural Networks and Learning Systems, 2019, 31(7): 2304-2314. [19] Song R, Chen L W, Yuan Q. Person name recognition method based on corpus and rule[C]//Proceedings of the Computational Language Research and Development, 1993. [20] Tan H Y, Huang C N, Dong Z D. Chinese place automatic recognition research[C]//Proceedings of Computational Language, 1999. [21] Luo Z, Song R. Integrated and fast recognition of proper noun in modern Chinese word segmentation[C]//Proceedings of International Conference on Chinese Computing, 2001: 323-328. [22] Fukuda K,Tsunoda T, Tamura A, et al. Toward information extraction: identifying protein names from biological papers[J].Pac symp biocomput, 1998, 707(18): 707-718. [23] Tsai R T H, Wu S H, Lee C W, et al. Mencius: A Chinese named entity recognizer using the maximum entropy-based hybrid model[J].International Journal of Computational Linguistics and Chinese Language Processing, 2004: 65-82. [24] Fang X, Sheng H. A hybrid approach forchinese named entity recognition[J]. International Conference on Discovery Science. Springer, 2002: 297-301. [25] Eddy S R. Hiddenmarkov models[J]. Current opinion in structural biology, 1996, 6(3): 361-365. [26] Kapur J N. Maximum-entropy models in science and engineering[M]. John Wiley and Sons, 1989. [27] Quinlan J R. Induction of decision trees[J]. Machine learning, 1986, 1(1): 81-106. [28] Hearst M A,Dumais S T, Osuna E, et al. Support vector machines[J]. IEEE Intelligent Systems and their applications, 1998, 13(4): 18-28. [29] Lafferty J, McCallum A, Pereira F C N. Conditional random fields: Probabilistic models for segmenting and labeling sequence data[C]//Proceedings of the 18th International Conference on Machine Learning. 2001: 282-289. [30] Fu G, Luke K K. Chinese named entity recognition using lexicalized HMMs[J]. ACM SIGKDD Explorations Newsletter, 2005, 7(1): 19-25. [31] Chen A, Peng F, Shan R, et al. Chinese named entity recognition with conditional probabilistic models[C]//Proceedings of the 5th SIGHAN Workshop on Chinese Language Processing, 2006: 173-176. [32] Yu X, Lam W, Chan S K, et al. Chinese NER using CRFs and logic for the fourth SIGHAN bakeoff[C]//Proceedings of the 6th SIGHAN Workshop on Chinese Language Processing, 2008. [33] Zhang Y, Xu Z, Zhang T. Fusion of multiple features forchinese named entity recognition based on CRF model[C]//Proceedings of the 4th Asia Information Retrieval Symposium, 2008: 95-106. [34] Settles B. Biomedical named entity recognition using conditional random fields and rich feature sets[C]//Proceedings of the International Joint Workshop on Natural Language Processing in Biomedicine and its Applications, 2004: 107-110. [35] Liu S, Sun Y, Li B, et al. HAMNER: Headword amplified multi-span distantly supervised method for domain specific named entity recognition[C]//Proceedings of the AAAI Conference on Artificial Intelligence, 2020: 8401-8408. [36] 张朝胜,郭剑毅,线岩团,等.基于条件随机场的英文产品命名实体识别[J].计算机工程与科学, 2010, 32(6): 115-117. [37] 郭剑毅,薛征山,余正涛,等.基于层叠条件随机场的旅游领域命名实体识别[J].中文信息学报, 2009, 23(5): 47-53. [38] Hammerton J. Named entity recognition with long short-term memory[C]//Proceedings of the 7th Conference on Natural Language Learning, 2003: 172-175. [39] Cho K, Van Merrinboer B, Bahdanau D, et al. On the properties of neural machine translation: Encoder-decoder approaches[J]. arXiv preprint arXiv:1409.1259, 2014. [40] Dong C, Zhang J,Zong C, et al. Character-based LSTM-CRF with radical-level features for Chinese named entity recognition[M]. Kunming: Springer, 2016: 239-250. [41] Jia Y, Ma X. Attention in Character-Based BiLSTM-CRF for Chinese named entity recognition[C]//Proceedings of the 4th International Conference on Mathematics and Artificial Intelligence, 2019: 1-4. [42] Cao P, Chen Y, Liu K, et al. Adversarial transfer learning for Chinese named entity recognition with self-attention mechanism[C]//Proceedings of the Conference on Empirical Methods in Natural Language Processing, 2018: 182-192. [43] Wu F, Liu J, Wu C, et al. Neural Chinese named entity recognition via CNN-LSTM-CRF and joint training with word segmentation[C]//Proceedings of the The World Wide Web Conference, 2019: 3342-3348. [44] Liu W, Xu T, Xu Q, et al. An encoding strategy based word-character LSTM for Chinese NER[C]//Proceedings of the Conference of the North American Chapter of the Association for Computational Linguistics, 2019: 2379-2389. [45] Wang Q, Zhou Y,Ruan T, et al. Incorporating dictionaries into deep neural networks for the Chinese clinical named entity recognition[J]. Journal of biomedical informatics, 2019, 92: 103133. [46] Xu C, Wang F, Han J, et al. Exploiting multiple embeddings for Chinese named entity recognition[C]//Proceedings of the 28th ACM International Conference on Information and Knowledge Management, 2019: 2269-2272. [47] Zhu Y, Wang G, Karlsson B F. CAN-NER: Convolutional attention network for Chinese named entity recognition[J].arXiv preprint arXiv:1904.02141, 2019. [48] Collobert R, Weston J. A unified architecture for natural language processing: Deep neural networks with multitask learning[C]//Proceedings of the 25th international conference on Machine learning, 2008: 160-167. [49] Collobert R, Weston J, Bottou L, et al. Natural language processing (almost) from scratch[J]. Journal of machine learning research, 2011, 12: 2493-2537. [50] Kalchbrenner N, Grefenstette E, Blunsom P. A convolutional neural network for modelling sentences[J]. arXiv preprint arXiv:1404.2188, 2014. [51] Kim Y. Convolutional neural networks for sentence classification[J].arXiv preprint arXiv:1408.5882, 2014. [52] Strubell E, Verga P, Belanger D, et al. Fast and accurate entity recognition with iterated dilated convolutions[J]. arXiv preprint arXiv:1702.02098, 2017. [53] Gui T, Ma R, Zhang Q, et al. CNN-Based Chinese NER with lexicon rethinking[C]//Proceedings of the 28th International Joint Conference on Artificial Intelligence, 2019: 4982-4988. [54] Kipf T N, Welling M. Semi-supervised classification with graph convolutional networks[J]. arXiv preprint arXiv:1609.02907, 2016. [55] Velicˇkovic＇ P, Cucurull G, Casanova A, et al. Graph attention networks[J]. arXiv preprint arXiv:1710.10903, 2017. [56] Li Y,Tarlow D, Brockschmidt M, et al. Gated graph sequence neural networks[J]. arXiv preprint arXiv:1511.05493, 2015. [57] Sui D, Chen Y, Liu K, et al. Leverage lexical knowledge forchinese named entity recognition via collaborative graph network[C]//Proceedings of the Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing, 2019: 3821-3831. [58] Gui T, Zou Y, Zhang Q, et al. A lexicon-based graph neural network for Chinese ner[C]//Proceedings of the Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing, 2019: 1039-1049. [59] Vaswani A,Shazeer N, Parmar N, et al. Attention is all you need[C]//Proceedings of the 31st International Conference on, 2017: 5998-6008. [60] Li X, Yan H,Qiu X, et al. FLAT: Chinese NER using flat-lattice transformer[J]. arXiv preprint arXiv:2004.11795, 2020. [61] Mengge X, Bowen Y, Tingwen L, et al. Porous lattice-based transformer encoder for Chinese NER[J]. arXiv preprint arXiv:1911.02733, 2019. [62] Peng M, Ma R, Zhang Q, et al. Simplify the usage of lexicon in Chinese NER[J].arXiv preprint arXiv:1908.05969, 2019. [63] Hu M, Peng Y, Huang Z, et al. Open-domain targeted sentiment analysis via span-based extraction and classification[J].arXiv preprint arXiv:1906.03820, 2019. [64] Sun L, Sun Y, Ji F, et al. Joint learning of token context and span feature for span-based nested NER[J]. IEEE/ACM Transactions on Audio, Speech, and Language Processing, 2020, 28: 2720-2730. [65] Dong X, Xin X, Guo P.Chinese NER by span-level self-attention[C]// Proceedings of the 15th International Conference on Computational Intelligence and Security, 2019: 68-72. [66] Xu M, Jiang H,Watcharawittayakul S. A local detection approach for named entity recognition and mention detection[C]//Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics, 2017: 1237-1247. [67] Sohrab M G, Miwa M. Deep exhaustive model for nested named entity recognition[C]//Proceedings of the Conference on Empirical Methods in Natural Language Processing, 2018: 2843-2849. [68] Mao H, Tang H, Zhang W, et al. A span-based distantly supervised NER with self-learning[C]// Proceedings of the CCF International Conference on Natural Language Processing and Chinese Computing, 2020: 192-203. [69] Wang S, Jiang J. Machine comprehension using match-lstm and answer pointer[J]. arXiv preprint arXiv:1608.07905, 2016. [70] Li X, Feng J, Meng Y, et al. A unifiedmrc framework for named entity recognition[J]. arXiv preprint arXiv:1910.11476, 2019. [71] Li S, Zhao Z, Hu R, et al. Analogical reasoning onchinese morphological and semantic relations[J]. arXiv preprint arXiv:1805.06504, 2018. [72] Katiyar A, Cardie C. Nested named entity recognition revisited[C]//Proceedings of the Conference of the North American Chapter of the Association for Computational Linguistics, 2018: 861-871. [73] Straková J, Straka M, Hajicˇ J. Neural architectures for nested NER through linearization[J]. arXiv preprint arXiv:1908.06926, 2019. [74] Ju M, Miwa M,Ananiadou S. A neural layered model for nested named entity recognition[C]//Proceedings of the Conference of the North American Chapter of the Association for Computational Linguistics, 2018: 1446-1459. [75] Jue W, Shou L, Chen K, et al. Pyramid: A layered model for nested named entity recognition[C]//Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, 2020: 5918-5928. [76] Lin H, Lu Y, Han X, et al. Sequence-to-nuggets: Nested entity mention detection via anchor-region networks[J].arXiv preprint arXiv:1906.03783, 2019. [77] Zhou G, Zhang J,Su J, et al. Recognizing names in biomedical texts: A machine learning approach[J]. Bioinformatics, 2004, 20(7): 1178-1190. [78] Zhou G D. Recognizing names in biomedical texts using mutual information independence model and SVM plus sigmoid[J]. International Journal of Medical Informatics, 2006, 75(6): 456-467. [79] 刘非凡,赵军,徐波.实体提及的多层嵌套识别方法研究[J].中文信息学报, 2007, 21(2): 14-21. [80] Fu C, Fu G. Morpheme-basedchinese nested named entity recognition[C]//Proceedings of the 8th International Conference on Fuzzy Systems and Knowledge Discovery, 2011, 2: 1221-1225. [81] Fu C, Fu G. A dual-layer CRFs based method forchinese nested named entity recognition[C]//Proceedings of the 9th International Conference on Fuzzy Systems and Knowledge Discovery, 2012: 2546-2550. [82] 郭慧志,刘华,谢学敏,等.《人民日报》标注语料的初步统计分析[C]//全国第八届计算语言学联合学术会议(JSCL-2005)论文集, 2005. [83] 黄鑫,朱巧明,钱龙华,等.基于特征组合的中文实体关系抽取[J].微电子学与计算机, 2010, 27(4): 198-200. [84] 徐薇,付滨,刘柳,等.中文命名实体识别系统的领域扩展[C]//第9届全国计算语言学学术会议论文集, 2007. [85] Fu G, Luke K K. Chinese named entity recognition using lexicalized HMMs[J]. ACM SIGKDD Explorations Newsletter, 2005, 7(1): 19-25. [86] Chen H H, Ding Y W, Tsai S C, et al. Description of the NTU system used for MET-2[C]//Proceedings of the 7th Message Understanding Conference, 1998: 1-9. [87] 李雁群,何云琪,钱龙华,等.中文嵌套命名实体识别语料库的构建[J].中文信息学报, 2018, 32(8): 19-26. [88] 李雁群,何云琪,钱龙华,等.基于维基百科的中文嵌套命名实体识别语料库自动构建[J].计算机工程, 2018: 12. [89] 许浩亮,李雁群,何云琪,等.中文嵌套命名实体关系抽取研究[J].北京大学学报(自然科学版), 2019(01): 8-14.
2022, Vol. 36 
