现有的命名实体识别方法主要是将句子看作一个序列进行处理,忽略了句子中潜在的句法信息,存在长距离依赖问题。为此,该文提出一种基于依存关系的命名实体识别模型,通过在输入数据中增加依存树信息,改变双向长短时记忆网络的层间传播方式,以获得单词在依存树中的子节点和父节点信息,并通过注意力机制动态选择两者的特征,最后将特征输入到CRF层实现命名实体标注。实验表明,该方法较BiLSTM-CRF模型在性能上得到了提高,且在长实体识别上优势明显。在OntoNotes 5.0 English和OntoNotes 5.0 Chinese以及SemEval-2010 Task 1 Spanish上的F1值分别达到了88.94%、77.42%、84.38%。
Abstract
Most of the existing named entity recognition methods treat the sentence as a sequence, ignoring the syntactic information in the sentence. This paper proposed a named entity recognition model based on dependency relationship. Adding dependency tree information to the input data, the child and parent node information of words in the dependency tree are obtained by changing the inter layer propagation mode in Bi-LSTM. The features are dynamically selected by the attention mechanism. Finally, the CRF layer is adopted to realize named entity annotation. Experimental results show that the proposed method is better than BiLSTM-CRF model, especially for long entity recognition, achieving 88.94%, 77.42% and 84.38% F1 values on OntoNotes 5.0 English, OntoNotes 5.0 Chinese and semeval-2010 Task 1 Spanish respectively.
关键词
命名实体识别 /
依存树 /
有向图 /
注意力机制
{{custom_keyword}} /
Key words
named entity recognition /
dependency tree /
directed-graph /
attention mechanism
{{custom_keyword}} /
{{custom_sec.title}}
{{custom_sec.title}}
{{custom_sec.content}}
参考文献
[1] Huang Z,Xu W,Yu K. Bidirectional LSTM-CRF models for sequence tagging[J]. arXiv preprint arXiv: 1508.01991, 2015.
[2] 殷章志,李欣子,黄德根,等. 融合字词模型的中文命名实体识别研究[J]. 中文信息学报, 2019, 33(11): 95-100,106.
[3] 林广和,张绍武,林鸿飞. 基于细粒度词表示的命名实体识别研究[J]. 中文信息学报, 2018, 32(11): 62-71,78.
[4] Peters M, Neumann M, Iyyer M, et al. Deep contextualized word representations[C]//Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, 2018: 2227-2237.
[5] Devlin J, Chang M W, Lee K, et al. Bert: Pre-training of deep bidirectional transformers for language understanding[J]. arXiv preprint arXiv: 1810.04805, 2018.
[6] Mikolov T, Martin Karafiát, Burget L, et al. Recurrent neural network based language model[C]//Proceedings of the 11th Annual Conference of the International Speech Communication Association, 2010: 2366-2370.
[7] Hochreiter S, Schmidhuber J. Long Short-Term Memory[J]. Neural Computation, 1997, 9(8): 1735-1780.
[8] Sasano R and Kurohashi S. Japanese named entity recognition using structural natural language processing[C]//Proceedings of the 3rd International Joint Conference on Natural Language Processing, 2008: 607-612.
[9] Ling X, Weld D. Fine-grained entity recognition[C]//Proceedings of the 26th Conference on Artificial Intelligence, 2012: 8008-8009.
[10] Jie Z, Muis A, Lu W. Effificient dependency-guided named entity recognition[C]//Proceedings of the 31st AAAI Conference on Artificial Intelligence, 2017: 3457-3465.
[11] Miwa M, Bansal M.End-to-end relation extraction using LSTMs on sequences and tree structures[C]//Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics, 2016: 1105-1116.
[12] Jie Z, Lu W.Dependency-guided LSTM-CRF for named entity recognition[C]//Proceedings of Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing, 2018: 3862-3872.
[13] Rei M, Crichton G K O, Pyysalo S . Attending to characters in neural sequence labeling models[C] //Proceedings of the 26th International Conference on Computational Linguistics: Technical Papers, 2016: 309-318.
[14] Ma X, Hovy E. End-to-end sequence labeling via bi-directional LSTM-CNNs-CRF[C]//Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics, 2016: 1064-1074.
[15] Hovy E, Marcus M, Palmer M, et al. OntoNotes: the 90% solution[C]//Proceedings of the Human Language Technology Conference of the NAACL, 2006: 57-60.
[16] Recasens M, Toni Martí, Mariona Taulé, et al. SemEval-2010 Task 1: Coreference resolution in multiple languages[C]//Proceedings of the NAACL HLT Workshop on Semantic Evaluations: Recent Achievements and Future Directions, 2009: 70-75.
[17] Manning C D, Surdeanu M, Bauer J, et al.The stanford corenlp natural language processing toolkit[C]//Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics: System Demonstrations, 2014: 55-60.
[18] Pradhan S, Moschitti A, Xue N, et al. Towards robust linguistic analysis using ontonotes[C]//Proceedings of the 17th Conference on Computational Natural Language Learning,2013: 143-152.
[19] Pradhan S, Moschitti A, Xue N, et al.Conll-2012 shared task: Modeling multilingual unrestricted coreference in ontonotes[C]//Proceedings of Joint Conference on EMNLP and CoNLL - Shared Task, 2012: 1-40.
[20] Finkel J R, Manning C D. Joint parsing and named entity recognition[C]//Proceedings of Human Language Technologies: The 2009 Annual Conference of the North American Chapter of the Association for Computational Linguistics, 2009: 326-334.
[21] Pennington J, Socher R, Manning C D. Glove: Global vectors for word representation[C]//Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP), 2014: 1532-1543.
[22] Grave E, Bojanowski P, Gupta P, et al. Learning word vectors for 157 languages[C]//Proceedings of the 11th International Conference on Language Resources and Evaluation, 2018: 3483-3487.
[23] Li P H, Dong R P, Wang Y S, et al. Leveraging linguistic structures for named entity recognition with bidirectional recursive neural networks[C]//Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, 2017: 2664-2669.
[24] Ghaddar A, Langlais P.Robust lexical features for improved neural network named-entity recogni-tion[C]//Proceedings of the 27th International Conference on Computational Linguistics, 2018: 1896-1907.
{{custom_fnGroup.title_cn}}
脚注
{{custom_fn.content}}
基金
国家自然科学基金(61672127,U1936109);河北省重点研发计划项目民生专项(20375701D)
{{custom_fund}}
PDF(3263 KB)
