Hierarchical Multi-modal Fusion and Difficult Negative Sampling Knowledge Graph Completion

Multimodal knowledge graph completion aims to fuse text, image and structure information perfect a knowledge graph. To deal with challenges of the modal heterogeneity, the insufficient cross-modal interaction and the random negative samples, this paper proposes a Hierarchical Multimodal Fusion and Hard Negative Sampling (HM-HNS) framework. First, customized processing modules are designed to capture different modal characteristics. Then, a hierarchical fusion strategy is adopted to achieve fine-grained cross-modal alignment. Finally, a dynamic hard sample generation mechanism is employed to significantly improve negative sample quality. Experiments demonstrate that this model obtains relatively better performance on multiple public datasets.