Distant Supervision for Tibetan Entity Relation Extraction
WANG Like1,2, SUN Yuan1,2, XIA Tianci1,2
1.School of Information Engineering, Minzu University of China, Beijing 100081, China; 2.Minority Languages Branch, National Language Resource and Monitoring Research Center, Minzu University of China, Beijing 100081, China
Abstract：Distant supervision for relation extraction is an efficient method to automatically align entities in texts to a given knowledge base (KB), which alleviated the problem of manual labelling. In this paper, we propose an improved distant supervised relation extraction model in Tibetan based on Piecewise Convolutional Neural Network (PCNN). The language model and the selective-attention mechanism are combined to alleviate wrong labelling problems and to extract effective features. Soft-label method is also introduced to dynamically correct the relation label. The experimental results show that our method is effective and outperforms several competitive baseline methods.
[1] M. Mintz, S. Bills, R. Snow, et al. Distant supervision for relation extraction without labeled data[C]// Proceedings of the 47th Annual Meeting of the Association for Computational Linguistics and the 4th International Joint Conference on Natural Language Processing of the AFNLP, 2009: 1003-1011. [2] T. Mikolov, K. Chen, G. Corrado, et al. Efficient estimation of word representations in vector space[J]. arXiv preprint arXiv: 1301.3781, 2013. [3] E Grave, T Mikolov, A Joulin, et al. Bag of tricks for efficient text classification[J]. arXiv preprint arXiv: 1607.01759, 2016. [4] J P Turian, L A Ratinov, Y Bengio. Word representations: A simple and general method for semi-supervised learning[C]// Proceedings of the 48th Annual Meeting of the Association for Computational Linguistics, 2010: 384-394. [5] T Mikolov, I Sutskever, K Chen, et al. Distributed representations of words and phras es and their compositionality[J]. Advances in Neural Information Processing Systems, 2013, 26: 3111-3119. [6] A Neelakantan, J Shankar, A Passos, et al. Efficient non-parametric estimation of multiple embeddings per word in vector space[C]// Proceedings of the Empirical Methods in Natural Language Processing, 2015: 1059-1069. [7] J Wieting, M Bansal, K Gimpel, et al. Charagram: Embedding words and sentences via character n-grams[C]// Proceedings of the Empirical Methods in Natural Language Processing,2016: 1504-1515. [8] P Bojanowski, E Grave, A Joulin, et al. Enriching word vectors with subword information[J]. Transactions of the Association for Computational Linguistics, 2017, 5: 135-146. [9] O Melamud, J Goldberger, I Dagan. Context2vec: Learning generic context embedding with bidirectional LSTM[C]// Proceedings of the 20th SIGNLL Conference on Computational Natural Language Learning. 2016: 51-61. [10] B Mccann, J Bradbury, C M. Xiong, et al. Learned in translation: Contextualized word vectors[C]// Proceedings of the Neural Information Processing Systems, 2017: 6294-6305. [11] M E Peters, W Ammar, C Bhagavatula, et al. Semi-supervised sequence tagging with bidirectional language models[C]// Proceedings of the Association for Computational Linguistics, 2017: 1756-1765. [12] M E Peters, M Neumann, M Iyyer, et al. Deep contextualized word representations[C]// Proceedings of the Association for Computational Linguistics, 2018: 2227-2237. [13] Y Bengio. Learning Deep architectures for AI[J]. Foundations and Trends R in Machine Learning, 2009, 2(1): 1-127. [14] R Collobert, J Weston, L Bottou, et al. Natural language processing (almost) from scratch[J]. Journal of Machine Learning Research, 2011, 12(1): 2493-2537. [15] C N dos Santos, M A de C Gatti. Deep convolutional neural networks for sentiment analysis of short texts[C]//Proceedings of the COLING 2014, the 25th International Conference on Computational Linguistics, 2014: 69-78. [16] R Socher, J Bauer, C D Manning, et al. Parsing with compositional vector grammars[C]// Proceedings of the Association for Computational Linguistics. 2013: 455-465. [17] I Sutskever, O Vinyals, Q V Le. Sequence to sequence learning with neural networks[J]. Advances in Neural Information Processing Systems, 2014, 2: 3104-3112. [18] J Hoffart, F M Suchanek, K Berberich, et al. YAGO2: A spatially and temporally enhanced knowledge base from Wikipedia[J]. Artificial Intelligence, 2013, 194: 28-61. [19] B Yue, M Gui, J H. Guo, et al. An effective framework for question answering over Freebase via reconstructing natural sequences[C]// Proceedings of The 26th International Conference on World Wide Web Companion. International World Wide Web Conferences Steering Committee, 2017: 865-866. [20] D Ritze, C Bizer. Matching web tables to DBpedia-a feature utility study[C]// Proceedings of the 20th EDBT, 2017: 210-221. [21] F A O Santos, F B do Nascimento, M I S Santos, et al. Training neural tensor networks with the never ending language learner[M]. Information Technology-New Generations. Cham: Springer, 2018: 19-23. [22] D J Zeng, K Liu, Y B Chen, et al. Distant supervision for relation extraction via piecewise convolutional, neural networks[C]// Proceedings of the Empirical Methods in Natural Language Processing, 2015: 1753-1762. [23] Y K Lin, S Q Shen, Z Y Liu, et al. Neural relation extraction with selective attention over instances[C]// Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics, 2016: 2124-2133. [24] X Jiang, Q Wang, P Li, et al. Relation extraction with multi-instance multi-label convolutional neural networks[C] //Proceedings of the COLING, 2016: 1471-1480. [25] X C Feng, J Guo, B Qin, et al. Effective deep memory networks for distant supervised relation extraction[C] //Proceedings of the Twenty-Sixth International Joint Conference on Artificial Intelligence. 2017: 4002-4008. [26] G L Ji, K Liu, S Z He, et al. Distant supervision for relation extraction with sentence-level attention and entity descriptions[C] // Proceedings of the AAAI, 2017: 3060-3066. [27] 朱臻, 孙媛. 基于SVM和泛化模板协作的藏语人物属性抽取[J]. 中文信息学报, 2015, 29(6): 220-227. [28] 郭莉莉,孙媛. 基于BP神经网络的藏语实体关系抽取[J/OL].软件导刊, 2019, 18(03): 13-15,21. [29] 夏天赐,孙媛. 基于联合模型的藏文实体关系抽取方法研究[J]. 中文信息学报, 2018,32(12): 76-83. [30] S Riedel, L M Yao, A McCallum. Modeling relations and their mentions without labeled text[C]// Proceedings of The ECML PKDD, 2010: 148-163. [31] T Y Liu, K X Wang, B B Chang, et al. A soft-label method for noise-tolerant distantly supervised relation extraction[C]//Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, 2017: 1790-1795.
2020, Vol. 34 
