基于深度学习的关系抽取研究综述

庄传志,靳小龙,朱伟建,刘静伟,白龙,程学旗

PDF(5509 KB)
PDF(5509 KB)
中文信息学报 ›› 2019, Vol. 33 ›› Issue (12) : 1-18.
综述

基于深度学习的关系抽取研究综述

  • 庄传志1,2,靳小龙1,2,朱伟建1,2,刘静伟1,2,白龙1,2,程学旗1,2
作者信息 +

Deep Learning Based Relation Extraction: A Survey

  • ZHUANG Chuanzhi1,2, JIN Xiaolong1,2, ZHU Weijian1,2, LIU Jingwei1,2, BAI Long1,2, CHENG Xueqi1,2
Author information +
History +

摘要

关系抽取(RE)是为了抽取文本中包含的关系,是信息抽取(IE)的重要组成部分。近年来,研究人员利用深度学习技术在该领域开展了深入研究。由于神经网络类型丰富,基于深度学习的关系抽取方法也更加多样。该文从关系抽取的基本概念出发,对关系抽取方法依据不同的视角进行了类别划分。随后,介绍了基于深度学习的关系抽取方法常用的数据集,并总结出基于深度学习的关系抽取框架。在此框架下,对关系抽取方法在面向深度学习的输入数据预处理、面向深度学习的神经网络模型设计等方面的具体工作进行了分析与评述,最后对未来的研究方向进行了探讨和展望。

Abstract

Information Extraction(IE) is a task of natural language processing that involves extracting structured information from plain unstructured text. Relation Extraction(RE) is a crucial component in IE. Recently, researchers pay great attention to the method of deep learning, resulting various methods in this field. Starting from the basic concept of relationship extraction, this paper groupes methods from different perspectives, introduces the popular data sets, and outlines the deep learning framework for relationship extraction. This paper analyzes and reviews the details in data preprocessing and model design in these methods. Finally, the future research direction is discussed.

关键词

关系抽取 / 深度学习 / 远程监督 / 联合学习

Key words

relation extraction / deep learning / distant supervision / joint learning

引用本文

导出引用
庄传志,靳小龙,朱伟建,刘静伟,白龙,程学旗. 基于深度学习的关系抽取研究综述. 中文信息学报. 2019, 33(12): 1-18
ZHUANG Chuanzhi, JIN Xiaolong, ZHU Weijian, LIU Jingwei, BAI Long, CHENG Xueqi. Deep Learning Based Relation Extraction: A Survey. Journal of Chinese Information Processing. 2019, 33(12): 1-18

参考文献

[1] Wu F, Weld D S. Open information extraction using Wikipedia[C]//Proceedings of the 48th Annual Meeting of the Association for Computational Linguistics. Association for Computational Linguistics, 2010: 118-127.
[2] Fleischman M, Hovy E, Echihabi A. Offline strategies for online question answering: Answering questions before they are asked[C]//Proceedings of the 41st Annual Meeting on Association for Computational Linguistics-Volume 1. Association for Computational Linguistics, 2003: 1-7.
[3] Goodfellow I, Bengio Y, Courville A. Deep learning[M]. MIT press, 2016.
[4] Plank B, Sgaard A, Goldberg Y. Multilingual part-of-speech tagging with bidirectional long short-term memory models and auxiliary loss[J]. arXiv preprint arXiv: 1604.05529, 2016.
[5] Lee J Y, Dernoncourt F, Szolovits P. Transfer learning for named-entity recognition with neural networks[J]. arXiv preprint arXiv: 1705.06273, 2017.
[6] Zhou J, Xu W. End-to-end learning of semantic role labeling using recurrent neural networks[C]//Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing, 2015, 1: 1127-1137.
[7] Zeng D, Liu K, Lai S, et al. Relation classification via convolutional deep neural network[C]//Proceedings of the COLING 2014, the 25th International Conference on Computational Linguistics: Technical Papers. 2014: 2335-2344.
[8] Socher R, Huval B, Manning C D, et al. Semantic compositionality through recursive matrix-vector spaces[C]//Proceedings of the 2012 Joint Conference on Empirical Methods in Natural Language Processing and Computational Natural Language Learning. Association for Computational Linguistics, 2012: 1201-1211.
[9] Xu K, Feng Y, Huang S, et al. Semantic relation classification via convolutional neural networks with simple negative sampling[J]. arXiv preprint arXiv: 1506.07650, 2015.
[10] Xu Y, Jia R, Mou L, et al. Improved relation classification by deep recurrent neural networks with data augmentation[J]. arXiv preprint arXiv: 1601.03651, 2016.
[11] Zheng S, Wang F, Bao H, et al. Joint extraction of entities and relations based on a novel tagging scheme[J]. arXiv preprint arXiv: 1706.05075, 2017.
[12] Kumar S. A survey of deep learning methods for relation extraction[J]. arXiv preprint arXiv: 1705.03645, 2017.
[13] 徐建,张智雄,吴振新,实体关系抽取的技术方法综述[J]. 现代图书情报技术,2008,168(8): 18-23.
[14] 刘方驰,钟志农,雷霖,等. 基于机器学习的实体关系抽取方法[J]. 兵工自动化, 2013, 32(9): 57-62.
[15] Xun H, Hongliang Y, Yang Y. A review of relation extraction[J]. Data Analysis and Knowledge Discovery, 2013, 29(11): 30-39.
[16] Mintz M, Bills S, Snow R, et al. Distant supervision for relation extraction without labeled data[C]//Proceedings of the Joint Conference of the 47th Annual Meeting of the ACL and the 4th International Joint Conference on Natural Language Processing of the AFNLP: Volume 2. Association for Computational Linguistics, 2009: 1003-1011.
[17] Brin S. Extracting patterns and relations from the world wide web[C]//Proceedings of the International Workshop on The World Wide Web and Databases. Springer, Berlin, Heidelberg, 1998: 172-183.
[18] Hasegawa T, Sekine S, Grishman R. Discovering relations among named entities from large corpora[C]//Proceedings of the 42nd Annual Meeting on Association for Computational Linguistics. Association for Computational Linguistics, 2004: 415.
[19] Zhao S, Grishman R. Extracting relations with integrated information using kernel methods[C]//Proceedings of the 43rd Annual Meeting on Association for Computational Linguistics. Association for Computational Linguistics, 2005: 419-426.
[20] Kambhatla N. Combining lexical, syntactic, and semantic features with maximum entropy models for extracting relations[C]//Proceedings of the ACL 2004 on Interactive Poster and Demonstration Sessions. Association for Computational Linguistics, 2004: 22.
[21] Sam R C, Le H T, Nguyen T T. Relation extraction in Vietnamese text using conditional random fields[C]//Proceedings of Asia Information Retrieval Symposium. Springer, Berlin, Heidelberg, 2010: 330-339.
[22] Zelenko D, Aone C, Richardella A. Kernel methods for relation extraction[J]. Journal of Machine Learning Research, 2003, 3(Feb): 1083-1106.
[23] Katiyar A, Cardie C. Going out on a limb: Joint extraction of entity mentions and relations without dependency trees[C]//Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics, 2017, (1): 917-928.
[24] Doddington G R, Mitchell A, Przybocki M A, et al. The automatic content extraction (ACE) program-tasks, data, and evaluation[C]//Proceedings of the 4th International Conference on Language Resources and Evaluation, 2004: 837-840.
[25] Hendrickx I, Kim S N, Kozareva Z, et al. Semeval-2010 task 8: Multi-way classification of semantic relations between pairs of nominals[C]//Proceedings of the Workshop on Semantic Evaluations: Recent Achievements and Future Directions. Association for Computational Linguistics, 2009: 94-99.
[26] Riedel S, Yao L, McCallum A. Modeling relations and their mentions without labeled text[C]//Proceedings of the Joint European Conference on Machine Learning and Knowledge Discovery in Databases. Springer, Berlin, Heidelberg, 2010: 148-163.
[27] Zhang D, Wang D. Relation classification via recurrent neural network[J]. arXiv preprint arXiv: 1508.01006, 2015.
[28] Santos C N, Xiang B, Zhou B. Classifying relations by ranking with convolutional neural networks[J]. arXiv preprint arXiv: 1504.06580, 2015.
[29] Rink B, Harabagiu S. Utd: Classifying semantic relations by combining lexical and semantic resources[C]//Proceedings of the 5th International Workshop on Semantic Evaluation. Association for Computational Linguistics, 2010: 256-259.
[30] Hou X, Zhao Z, Liu T, et al. Classifying relation via bidirectional recurrent neural network based on local information[C]//Proceedings of the Asia-Pacific Web Conference. Springer, Cham, 2016: 420-430.
[31] Zheng S, Xu J, Zhou P, et al. A neural network framework for relation extraction: Learning entity semantic and relation pattern[J]. Knowledge-Based Systems, 2016, 114: 12-23.
[32] Qin P, Xu W, Guo J. An empirical convolutional neural network approach for semantic relation classification[J]. Neurocomputing, 2016, 190: 1-9.
[33] Liu C Y, Sun W B, Chao W H, et al. Convolution neural network for relation extraction[C] //Proceedings of the International Conference on Advanced Data Mining and Applications. Springer, Berlin, Heidelberg, 2013: 231-242.
[34] Hashimoto K, Miwa M, Tsuruoka Y, et al. Simple customization of recursive neural networks for semantic relation classification[C]//Proceedings of the 2013 Conference on Empirical Methods in Natural Language Processing, 2013: 1372-1376.
[35] Yang Y, Tong Y, Ma S, et al. A position encoding convolutional neural network based on dependency tree for relation classification[C]//Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing, 2016: 65-74.
[36] Ebrahimi J, Dou D. Chain based RNN for relation classification[C]//Proceedings of the 2015 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, 2015: 1244-1249.
[37] Xu Y, Mou L, Li G, et al. Classifying relations via long short term memory networks along shortest dependency paths[C]//Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing, 2015: 1785-1794.
[38] Nguyen D, Matsuo Y, Ishizuka M. Relation extraction from wikipedia using subtree mining[C]//Proceedings of the 22nd National Conference on Artificial Intelligence-Volume 2. AAAI Press, 2007: 1414-1420.
[39] Liu Y, Wei F, Li S, et al. A dependency-based neural network for relation classification[J]. arXiv preprint arXiv: 1507.04646, 2015.
[40] Collobert R, Weston J, Bottou L, et al. Natural language processing (almost) from scratch[J]. Journal of Machine Learning Research, 2011, 12(Aug): 2493-2537.
[41] Zhang S, Zheng D, Hu X, et al. Bidirectional long short-term memory networks for relation classification[C]//Proceedings of the 29th Pacific Asia Conference on Language, Information and Computation. 2015: 73-78.
[42] Frege G. ber sinn und bedeutung[J]. Zeitschrift für Philosophie und philosophische Kritik, 1892, 100(1): 25-50.
[43] Mnih V, Heess N, Graves A. Recurrent models of visual attention[C]//Proceedings of the Advances in Neural Information Processing Systems, 2014: 2204-2212.
[44] Socher R, Pennington J, Huang E H, et al. Semi-supervised recursive autoencoders for predicting sentiment distributions[C]//Proceedings of the Conference on Empirical Methods in Natural Language Processing. Association for Computational Linguistics, 2011: 151-161.
[45] Nguyen T H, Grishman R. Relation extraction: Perspective from convolutional neural networks[C]//Proceedings of the 1st Workshop on Vector Space Modeling for Natural Language Processing, 2015: 39-48.
[46] Zeng D, Liu K, Chen Y, et al. Distant supervision for relation extraction via piecewise convolutional neural networks[C]//Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing, 2015: 1753-1762.
[47] Wang L, Cao Z, De Melo G, et al. Relation classification via multi-level attention cnns[C]// Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics. 2016: 1298-1307.
[48] Zhu J, Qiao J, Dai X, et al. Relation classification via target-concentrated attention cnns[C]//Proceedings of the International Conference on Neural Information Processing. Springer, Cham, 2017: 137-146.
[49] Lin Y, Shen S, Liu Z, et al. Neural relation extraction with selective attention over instances[C]//Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics, 2016, (1): 2124-2133.
[50] Ji G, Liu K, He S, et al. Distant supervision for relation extraction with sentence-level attention and entity descriptions[C]//Proceedings of the 31st AAAI Conference on Artificial Intelligence, 2017.
[51] Bordes A, Usunier N, Garcia-Duran A, et al. Translating embeddings for modeling multi-relational data[C]//Proceedings of the Advances in Neural Information Processing Systems, 2013: 2787-2795.
[52] 邱锡鹏. 神经网络与深度学习[EB/OL]. [2017-04-21]. https://nndl.github.io/.
[53] Mikolov T, Karafiát M, Burget L, et al. Recurrent neural network based language model[C]//Proceedings of the 11th Annual Conference of the International Speech Communication Association, 2010.
[54] Xiao M, Liu C. Semantic relation classification via hierarchical recurrent neural network with attention[C]//Proceedings of the COLING 2016, the 26th International Conference on Computational Linguistics: Technical Papers, 2016: 1254-1263.
[55] Hochreiter S, Schmidhuber J. Long short-term memory[J]. Neural Computation, 1997, 9(8): 1735-1780.
[56] Zhou P, Shi W, Tian J, et al. Attention-based bidirectional long short-term memory networks for relation classification[C]//Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics, 2016, (2): 207-212.
[57] Qin P, Xu W, Guo J. Designing an adaptive attention mechanism for relation classification[C]//Proceedings of the 2017 International Joint Conference on Neural Networks (IJCNN). IEEE, 2017: 4356-4362.
[58] Cho K, Van Merrinboer B, Gulcehre C, et al. Learning phrase representations using RNN encoder-decoder for statistical machine translation[J]. arXiv preprint arXiv: 1406.1078, 2014.
[59] Sun Y, Cui Y, Hu J, et al. Relation classification using coarse and fine-grained networks with SDP supervised key words selection[C]//Proceedings of the International Conference on Knowledge Science, Engineering and Management. Springer, Cham, 2018: 514-522.
[60] Vaswani A, Shazeer N, Parmar N, et al. Attention is all you need[C]//Proceedings of the Advances in Neural Information Processing Systems, 2017: 5998-6008.
[61] Lee J, Seo S, Choi Y S. Semantic relation classification via bidirectional LSTM networks with entity-aware attention using latent entity typing[J]. arXiv preprint arXiv: 1901.08163, 2019.
[62] Miwa M, Bansal M. End-to-end relation extraction using LSTMs on sequences and tree structures[J]. arXiv preprint arXiv: 1601.00770, 2016.
[63] Vu N T, Adel H, Gupta P, et al. Combining recurrent and convolutional neural networks for relation classification[J]. arXiv preprint arXiv: 1605.07333, 2016.
[64] Nguyen T H, Grishman R. Combining neural networks and log-linear models to improve relation extraction[J]. arXiv preprint arXiv: 1511.05926, 2015.
[65] Zhang X, Chen F, Huang R. A combination of RNN and CNN for attention-based relation classification[J]. Procedia Computer Science, 2018, 131: 911-917.
[66] Cai R, Zhang X, Wang H. Bidirectional recurrent convolutional neural network for relation classification[C]//Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics, 2016, (1): 756-765.
[67] Ren F, Zhou D, Liu Z, et al. Neural relation classification with text descriptions[C]//Proceedings of the 27th International Conference on Computational Linguistics, 2018: 1167-1177.
[68] Zhang Y, Qi P, Manning C D. Graph convolution over pruned dependency trees improves relation extraction[J]. arXiv preprint arXiv: 1809.10185, 2018.
[69] Zheng S, Hao Y, Lu D, et al. Joint entity and relation extraction based on a hybrid neural network[J]. Neurocomputing, 2017, 257: 59-66.
[70] Chen Y, Liu H, Wu C, et al. Distant supervision for relation extraction with neural instance selector[C]//Proceedings of the CCF International Conference on Natural Language Processing and Chinese Computing. Springer, Cham, 2018: 209-220.
[71] Jiang X, Wang Q, Li P, et al. Relation extraction with multi-instance multi-label convolutional neural networks[C]//Proceedings of the COLING 2016, the 26th International Conference on Computational Linguistics: Technical Papers, 2016: 1471-1480.
[72] Wang Z, Chang B, Sui Z. Distantly supervised neural network model for relation extraction[M].Chinese Computational Linguistics and Natural Language Processing Based on Naturally Annotated Big Data. Springer, Cham, 2015: 253-266.
[73] Feng X, Guo J, Qin B, et al. Effective deep memory networks for distant supervised relation extraction[C]//Proceedings of the IJCAI, 2017: 4002-4008.
[74] Liu T, Wang K, Chang B, et al. A soft-label method for noise-tolerant distantly supervised relation extraction[C]//Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, 2017: 1790-1795.
[75] Feng J, Huang M, Zhao L, et al. Reinforcement learning for relation classification from noisy data[C]//Proceedings of the 32nd AAAI Conference on Artificial Intelligence, 2018.
[76] Zeng X, He S, Liu K, et al. Large scaled relation extraction with reinforcement learning[C] //Proceedings of the 32nd AAAI Conference on Artificial Intelligence, 2018.
[77] Qin P, Xu W, Wang W Y. Dsgan: Generative adversarial training for distant supervision relation extraction[J]. arXiv preprint arXiv: 1805.09929, 2018.
[78] Wang S, Zhang Y, Che W, et al. Joint extraction of entities and relations based on a novel graph scheme[C]//Proceedings of the IJCAI, 2018: 4461-4467.
[79] Wu S, He Y. Enriching pre-trained language model with entity information for relation classification[J]. arXiv preprint arXiv: 1905.08284, 2019.

基金

国家重点研发计划(2016YFB1000902);国家自然科学基金(61772501,61572473,61572469,91646120)
PDF(5509 KB)

1617

Accesses

0

Citation

Detail

段落导航
相关文章

/