高级检索

基于层次化多模态融合与困难负采样的知识图谱补全

Hierarchical Multi-modal Fusion and Difficult Negative Sampling Knowledge Graph Completion

  • 摘要: 多模态知识图谱补全旨在融合文本、图像和结构信息以提升推理能力,但现有方法面临三大挑战:模态异构性致细粒度对齐难;交互不足致特征交互弱与模态失衡;随机负样本低效限制模型优化。该文提出层次化多模态融合与困难负采样(Hierarchical Multi-modal Fusion with Hard Negative Sampling, HM-HNS)框架。首先针对不同模态特性设计定制化处理模块,有效缓解模态异构性问题;其次采用层次化融合策略实现跨模态交互,解决细粒度对齐与重要性分配失衡的难题;最后采用动态困难样本生成机制,显著提升负样本质量。实验表明,该文模型在多个公开数据集上均取得较好性能,最终实现了多模态知识图谱补全性能的系统性提升。

     

    Abstract: Multimodal knowledge graph completion aims to fuse text, image and structure information perfect a knowledge graph. To deal with challenges of the modal heterogeneity, the insufficient cross-modal interaction and the random negative samples, this paper proposes a Hierarchical Multimodal Fusion and Hard Negative Sampling (HM-HNS) framework. First, customized processing modules are designed to capture different modal characteristics. Then, a hierarchical fusion strategy is adopted to achieve fine-grained cross-modal alignment. Finally, a dynamic hard sample generation mechanism is employed to significantly improve negative sample quality. Experiments demonstrate that this model obtains relatively better performance on multiple public datasets.

     

/

返回文章
返回