知识表示学习在关系抽取、自动问答等自然语言处理任务中获得了广泛关注,该技术旨在将知识库中的实体与关系表示为稠密低维实值向量。然而,已有的模型在建模知识库中的三元组时,或是忽略三元组的邻域信息,导致无法处理关联知识较少的罕见实体,或是在引入邻域信息时不能自适应地为每个实体抽取最相关的邻节点属性,导致引入了冗余信息。基于以上问题,该文在知识表示模型TransE的基础上提出了聚合邻域信息的联合知识表示模型TransE-NA(neighborhood aggregation on TransE)。该模型首先根据实体的稀疏度确定其邻节点数量,然后根据实体的邻边关系选取对应邻节点上最相关的属性作为实体的邻域信息。在链接预测和三元组分类任务上的实验结果表明,该文的模型效果超越了基线模型,验证了该模型能有效聚合邻域信息,缓解数据稀疏问题,改善知识表示性能。
Abstract
Knowledge representation learning, which aims to encode entities and relations into a dense, real-valued and low-dimensional semantic space, has drawn massive attention in natural language processing tasks, such as relation extraction and question answering. To better capture the neighbor information, we propose a model named TransE-NA (Neighborhood Aggregation on TransE) based on TransE, which determines the number of neighbors according to sparse degrees of entities and then aggregates the most relevant attributes of neighbors according to the corresponding relations. Experimental results on link prediction and triplet classification show that our approach outperforms baselines, alleviating the data sparsity issue and improving the performance effectively.
关键词
知识表示 /
邻域信息 /
知识图谱
{{custom_keyword}} /
Key words
knowledge representation /
neighborhood aggregation /
knowledge graph
{{custom_keyword}} /
{{custom_sec.title}}
{{custom_sec.title}}
{{custom_sec.content}}
参考文献
[1] Bordes A, Weston J, Usunier N. Open question answering with weakly supervised embedding models[C]//Proceedings of the Joint European Conference on Machine Learning and Knowledge Discovery in Databases. Springer, Berlin, Heidelberg, 2014: 165-180.
[2] Hoffmann R, Zhang C, Ling X, et al. Knowledge-based weak supervision for information extraction of overlapping relations[C]//Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies-Volume 1. Association for Computational Linguistics, 2011: 541-550.
[3] Peng M, Huang J, Sun Z, et al. Improving distant supervision of relation extraction with unsupervised methods[C]//Proceedings of the International Conference on Web Information Systems Engineering. Springer, Cham, 2016: 561-568.
[4] Damljanovic D, Bontcheva K. Named entity disambiguation using linked data[C]//Proceedings of the 9th Extended Semantic Web Conference, 2012: 231-240.
[5] Bollacker K, Evans C, Paritosh P, et al. Freebase: A collaboratively created graph database for structuring human knowledge[C]//Proceedings of the 2008 ACM SIGMOD International Conference on Management of Data. ACM, 2008: 1247-1250.
[6] Miller G A. WordNet: A lexical database for English[J]. Communications of the ACM, 1995, 38(11): 39-41.
[7] Suchanek F M, Kasneci G, Weikum G. Yago: A core of semantic knowledge[C]//Proceedings of the 16th International Conference on World Wide Web. ACM, 2007: 697-706.
[8] Bordes A, Usunier N, Garcia-Duran A, et al. Translating embeddings for modeling multi-relational data[C]//Proceedings of the 26th International Conference on Neural Information Processing Systems, 2013: 2787-2795.
[9] Wang Z, Zhang J, Feng J, et al. Knowledge graph embedding by translating on hyperplanes[C]//Proceedings of the 28th AAAI Conference on Artificial Intelligence, 2014.
[10] Lin Y, Liu Z, Sun M, et al. Learning entity and relation embeddings for knowledge graph completion[C]//Proceedings of the 29th AAAI Conference on Artificial Intelligence, 2015.
[11] Ji G, He S, Xu L, et al. Knowledge graph embedding via dynamic mapping matrix[C]//Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing, 2015, 1: 687-696.
[12] Feng J, Huang M, Yang Y. GAKE: Graph aware knowledge embedding[C]//Proceedings of COLING 2016, the 26th International Conference on Computational Linguistics: Technical Papers, 2016: 641-651.
[13] Nguyen D Q, Sirts K, Qu L, et al. Neighborhood mixture model for knowledge base completion[J]. arXiv preprint arXiv: 1606.06461, 2016.
[14] Wang C C, Cheng P J. Translating representations of knowledge graphs with neighbors[C]//Proceedings of the 41st International ACM SIGIR Conference on Research and Development in Information Retrieval. ACM, 2018: 917-920.
[15] Ji G, Liu K, He S, et al. Knowledge graph completion with adaptive sparse transfer matrix[C]//Proceedings of the 30th AAAI Conference on Artificial Intelligence, 2016.
[16] Bordes A, Glorot X, Weston J, et al. Joint learning of words and meaning representations for open-text semantic parsing[C]//Proceedings of the 15th International Conference on Artificial Intelligence and Statistics, 2012: 127-135.
[17] Bordes A, Weston J, Collobert R, et al. Learning structured embeddings of knowledge bases[C]//Proceedings of the 25th AAAI Conference on Artificial Intelligence, 2011.
[18] Socher R, Chen D, Manning C D, et al. Reasoning with neural tensor networks for knowledge base completion[C]//Proceedings of the 26th International Conference on Neural Information Processing Systems, 2013: 926-934.
[19] Bordes A, Glorot X, Weston J, et al. A semantic matching energy function for learning with multi-relational data[J]. Machine Learning, 2014, 94(2): 233-259.
[20] Jenatton R, Roux N L, Bordes A, et al. A latent factor model for highly multi-relational data[C]//Proceedings of the 25th International Conference on Neural Information Processing Systems, 2012: 3167-3175.
[21] Sutskever I, Tenenbaum J B, Salakhutdinov R R. Modelling relational data using Bayesian clustered tensor factorization[C]//Proceedings of the NIPS2009, 2009: 1821-1828.
[22] Gao H, Shi J, Qi G, et al. Triple context-based knowledge graph embedding[J]. IEEE Access, 2018, 6: 58978-58989.
[23] Werbos P J. Backpropagation through time: What it does and how to do it[J]. Proceedings of the IEEE, 1990, 78(10): 1550-1560.
[24] Lin Y, Liu Z, Luan H, et al. Modeling relation paths for representation learning of knowledge bases[J]. arXiv preprint arXiv: 1506.00379, 2015.
[25] Xu J, Chen K, Qiu X, et al. Knowledge graph representation with jointly structural and textual encoding[J]. arXiv preprint arXiv: 1611.08661, 2016.
[26] Zhang W. Knowledge graph embedding with diversity of structures[C]//Proceedings of the 26th International Conference on World Wide Web Companion. International World Wide Web Conferences Steering Committee, 2017: 747-753.
{{custom_fnGroup.title_cn}}
脚注
{{custom_fn.content}}
基金
国家重点研发计划项目(2018YFC1604000,2018YFC1604003);国家自然科学基金(61772382)
{{custom_fund}}
PDF(1327 KB)
