基于模态相似性路径的统一多模态实体对齐

PDF(5937 KB)

中文信息学报 ›› 2024, Vol. 38 ›› Issue (6) : 34-44.

知识表示与知识获取

基于模态相似性路径的统一多模态实体对齐

朱柏霖,桂韬,张奇

作者信息 +

Universal Multi-modal Entity Alignment via Iteratively Fusing Modality Similarity Paths

ZHU Bolin, GUI Tao, ZHANG Qi

Author information +

History +

摘要

实体对齐(EA)的目标是从多个知识图谱(KG)中识别等价的实体对,并构建一个更全面、统一的知识图谱。大多数EA方法主要关注KG的结构模式,缺乏对多模态信息的探索。已有的一些多模态EA方法在这个领域做出了良好的尝试。但是,它们存在两个缺点: (1)针对不同模态信息采用复杂且不同的建模方式,导致模态建模不一致且建模低效; (2)由于EA中各模态间的异质性,模态融合效果往往不佳。为了解决这些挑战,该文提出了PathFusion,使用模态相似性路径作为信息载体,有效地合并来自不同模态的信息。在真实世界的数据集上的实验结果显示,与最先进的方法相比,PathFusion在Hits@1上提高了22.4%～28.9%,在MRR上提高了0.194～0.245,验证了PathFusion的优越性。

Abstract

The objective of Entity Alignment (EA) is to identify equivalent entity pairs from multiple Knowledge Graphs (KGs) and create a more comprehensive and unified KG. The majority of EA methods have primarily focused on the structural modality of KGs, lacking exploration of multi-modal information. A few multi-modal EA methods have made good attempts in this field. Still, they have two shortcomings: (1) inconsistent and inefficient modality modeling that designs complex and distinct models for each modality; (2) ineffective modality fusion due to the heterogeneous nature of modalities in EA. To tackle these challenges, we propose PathFusion, which effectively combines information from different modalities using the path as an information carrier. Experimental results on real-world datasets demonstrate the superiority of PathFusion over state-of-the-art methods, with 22.4%～28.9% absolute improvement on Hits@1, and 0.194～0.245 absolute improvement on MRR.

导出引用

朱柏霖,桂韬,张奇. 基于模态相似性路径的统一多模态实体对齐. 中文信息学报. 2024, 38(6): 34-44

ZHU Bolin, GUI Tao, ZHANG Qi. Universal Multi-modal Entity Alignment via Iteratively Fusing Modality Similarity Paths. Journal of Chinese Information Processing. 2024, 38(6): 34-44

参考文献

[1] CAO Y, LI X, LIU H, et al. Pay more attention to relation exploration for knowledge base question answering[J]. arXiv preprint arXiv: 2305.02118, 2023.
[2] ZHANG X, XU G, SUN Y, et al. Crowdsourcing learning as domain adaptation: A case study on named entity recognition[J]. arXiv preprint arXiv: 2105.14980, 2021.
[3] JIANG C, JIANG Y, WU W, et al. Modeling label correlations for ultra-fine entity typing with neural pairwise conditional random field[J]. arXiv preprint arXiv: 2212.01581, 2022.
[4] HU X, ZHANG C, MA F, et al. Semi-supervised relation extraction via incremental meta self-training[J]. arXiv preprint arXiv: 2010.16410, 2020.
[5] SUN Z, ZHANG Q, HU W, et al. A benchmarking study of embedding-based entity alignment for knowledge graphs[J]. arXiv preprint arXiv: 2003.07743, 2020.
[6] LI C, CAO Y, HOU L, et al. Semi-supervised entity alignment via joint knowledge embedding model and cross-graph model[C]//Proceedings of the Conference on Empirical Methods In Natural Language Processing and the 9th International Joint Conference on Natural Language Processing, 2019: 2723-2732.
[7] MAO X, WANG W, XU H, et al. Relational reflection entity alignment[C]//Proceedings of the 29th ACM International Conference on Information & Knowledge Management, 2020: 1095-1104.
[8] SUN Z, WANG C, HU W, et al. Knowledge graph alignment network with gated multi-hop neighborhood aggregation[C]//Proceedings of the AAAI Conference on Artificial Intelligence, 2020, 34(01): 222-229.
[9] SUN Z, CHEN M, HU W, et al. Knowledge association with hyperbolic knowledge graph embeddings[J]. arXiv preprint arXiv: 2010.02162, 2020.
[10] MAO X, WANG W, WU Y, et al. Boosting the speed of entity alignment 10×: Dual attention matching network with normalized hard sample mining[C]//Proceedings of the Web Conference, 2021: 821-832.
[11] LIU Z, CAO Y, PAN L, et al. Exploring and evaluating attributes, values, and structures for entity alignment[J]. arXiv preprint arXiv: 2010.03249, 2020.
[12] GAO Y, LIU X, WU J, et al. Clusterea: Scalable entity alignment with stochastic training and normalized mini-batch similarities[C]//Proceedings of the 28th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, 2022: 421-431.
[13] LIU F, CHEN M, ROTH D, et al. Visual pivoting for unsupervised entity alignment[C]//Proceedings of the AAAI Conference on Artificial Intelligence, 2021, 35(5): 4257-4266.
[14] TRISEDYA B D, QI J, ZHANG R. Entity alignment between knowledge graphs using attribute embeddings[C]//Proceedings of the AAAI Conference on Artificial Intelligence, 2019, 33(01): 297-304.
[15] XU C, SU F, LEHMANN J. Time-aware graph neural networks for entity alignment between temporal knowledge graphs[J]. arXiv preprint arXiv: 2203.02150, 2022.
[16] LIU X, WU J, LI T, et al. Unsupervised entity alignment for temporal knowledge graphs[J]. arXiv preprint arXiv: 2302.00796, 2023.
[17] LIN Z, ZHANG Z, WANG M, et al. Multi-modal contrastive representation learning for entity alignment[J]. arXiv preprint arXiv: 2209.00891, 2022.
[18] CHEN L, LI Z, XU T, et al. Multi-modal siamese network for entity alignment[C]//Proceedings of the 28th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, 2022: 118-126.
[19] SUN Z, HU W, LI C. Cross-lingual entity alignment via joint attribute-preserving embedding[C]//Proceedings of the Semantic Web-ISWC, Springer International Publishing, 2017: 628-644.
[20] LI Q, GUO S, LUO Y, et al. Attribute-consistent knowledge graph representation learning for multi-modal entity alignment[J]. arXiv preprint arXiv: 2304.01563, 2023.
[21] CHEN L, LI Z, WANG Y, et al. MMEA: Entity alignment for multi-modal knowledge graph[C]//Proceedings of the Knowledge Science, Engineering and Management. Springer International Publishing, 2020: 134-147.
[22] CHEN M, TIAN Y, YANG M, et al. Multilingual knowledge graph embeddings for cross-lingual knowledge alignment[J]. arXiv preprint arXiv: 1611.03954, 2016.
[23] ZHU H, XIE R, LIU Z, et al. Iterative entity alignment via joint knowledge embeddings[C]//Proceedings of the IJCAI, 2017, 17: 4258-4264.
[24] SUN Z, HU W, ZHANG Q, et al. Bootstrapping entity alignment with knowledge graph embedding[C]//Proceedings of the IJCAI, 2018, 18: 4396-4402.
[25] WANG Z, LV Q, LAN X, et al. Cross-lingual knowledge graph alignment via graph convolutional networks[C]//Proceedings of the Conference on Empirical Methods in Natural Language Processing, 2018: 349-357.
[26] MAO X, WANG W, XU H, et al. MRAEA: An efficient and robust entity alignment approach for cross-lingual knowledge graph[C]//Proceedings of the 13th International Conference on Web Search and Data Mining, 2020: 420-428.
[27] MAO X, WANG W, WU Y, et al. LightEA: A scalable, robust, and interpretable entity alignment framework via three-view label propagation[J]. arXiv preprint arXiv: 2210.10436, 2022.
[28] MAO X, MA M, YUAN H, et al. An effective and efficient entity alignment decoding algorithm via third-order tensor isomorphism[C]//Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics, 2022: 5888-5898.
[29] BORDES A, USUNIER N, GARCIA DURAN A, et al. Translating embeddings for modeling multi-relational data[C]//Proceedings of the International Conference on Neural Information Processing Systems, 2013: 2787-2795.
[30] KIPF T N, WELLING M. Semi-supervised classification with graph convolutional networks[J]. arXiv preprint arXiv: 1609.02907, 2016.
[31] YIN P, YAN X, ZHOU J, et al. DGI: Easy and efficient inference for GNNs[J]. arXiv preprint arXiv: 2211.15082, 2022.
[32] GE C, LIU X, CHEN L, et al. Make it easy: An effective end-to-end entity alignment framework[C]//Proceedings of the 44th International ACM SIGIR Conference on Research and Development in Information Retrieval, 2021: 777-786.
[33] MAO X, WANG W, WU Y, et al. From alignment to assignment: Frustratingly simple unsupervised entity alignment[J]. arXiv preprint arXiv: 2109.02363, 2021.
[34] LI Y, CHEN J, LI Y, et al. Vision, deduction and alignment: An empirical study on multi-modal knowledge graph alignment[C]//Proceedings of the ICASSP IEEE International Conference on Acoustics, Speech and Signal Processing. IEEE, 2023: 1-5.
[35] XU C, SU F, XIONG B, et al. Time-aware entity alignment using temporal relational attention[C]//Proceedings of the ACM Web Conference, 2022: 788-797.
[36] CHEN Z, CHEN J, ZHANG W, et al. Meaformer: Multi-modal entity alignment transformer for meta modality hybrid[C]//Proceedings of the 31st ACM International Conference on Multimedia, 2023: 3317-3327.
[37] SUN Z, CHEN M, HU W. Knowing the no-match: Entity alignment with dangling cases[J]. arXiv preprint arXiv: 2106.02248, 2021.
[38] CHEN M, SHI W, ZHOU B, et al. Cross-lingual entity alignment with incidental supervision[J]. arXiv preprint arXiv: 2005.00171, 2020.
[39] LIU Z, LIN Y, CAO Y, et al. Swin transformer: Hierarchical vision transformer using shifted windows[C]//Proceedings of the IEEE/CVF International Conference on Computer Vision, 2021: 10012-10022.
[40] LIU Y, OTT M, GOYAL N, et al. RoBERTa: A robustly optimized bert pretraining approach[J]. arXiv preprint arXiv: 1907.11692, 2019.
[41] ZENG W, ZHAO X, TANG J, et al. Collective entity alignment via adaptive features[C]//Proceedings of the IEEE 36th International Conference on Data Engineering. 2020: 1870-1873.
[42] CUTURI M. Sinkhorn distances: Lightspeed computation of optimal transport[C]//Proceedings of the International Conference on Neural Information Processing Systems, 2013: 2292-2300.
[43] LIU Y, LI H, GARCIA-DURAN A, et al. MMKG: Multi-modal knowledge graphs[C]//Proceedings of the Semantic Web: 16th International Conference, ESWC, 2019: 459-474.
[44] GARCA DURN A, DUMANCˇIC＇ S, Niepert M. Learning sequence encoders for temporal knowledge graph completion[J]. arXiv preprint arXiv: 1809.03202, 2018.
[45] SUCHANEK F M, KASNECI G, WEIKUM G. Yago: A core of semantic knowledge[C]//Proceedings of the International Conference on World Wide Web, 2007: 697-706.
[46] VRANDECˇIC＇ D, KRTZSCH M. Wikidata: A free collaborative knowledgebase[J]. Communications of the ACM, 2014, 57(10): 78-85.
[47] HE F, LI Z, QIANG Y, et al. Unsupervised entity alignment using attribute triples and relation triples[C]//Proceedings of the Database Systems for Advanced Applications: 24th International Conference, DASFAA, 2019: 367-382.
[48] PEI S, YU L, HOEHNDORF R, et al. Semi-supervised entity alignment via knowledge graph embedding with awareness of degree difference[C]//Proceedings of the International Conference on World Wide Web, 2019: 3130-3136.
[49] CAI L, MAO X, MA M, et al. A simple temporal information matching mechanism for entity alignment between temporal knowledge graphs[J]. arXiv preprint arXiv: 2209.09677, 2022.

PDF(5937 KB)

669

Accesses

Citation

Detail

段落导航

摘要
Abstract
关键词
Key words
引用本文
参考文献

选择文件类型/文献管理软件名称

选择包含的内容

摘要

Abstract

关键词

Key words

引用本文

{{custom_sec.title}}

{{custom_sec.title}}

参考文献

{{custom_fnGroup.title_cn}}

脚注

Published
2024-07-15
Issue Date
2024-07-17