高级检索

基于对抗网络与关系引导注意力的知识推理方法

Knowledge Reasoning Method Based on Adversarial Networks and Relation-guided Attention

  • 摘要: 为解决多模态知识推理任务存在的视觉模态信息缺失和多模态噪声问题,该文提出了一种基于对抗网络与关系引导注意力的知识推理方法。通过多模态对抗学习,利用已有的文本模态信息生成缺失的视觉模态特征,从而缓解模态缺失造成的信息不平衡和实体对齐问题;该文设计了一种关系引导注意力机制,用来捕获与关系上下文相关的重要多模态特征,进而减少多模态噪声,缓解因模态缺失导致的信息不完整、特征稀疏以及跨模态语义不一致问题引发的错误对齐与特征偏移问题,从而提升了模型在信息缺失与噪声扰动下的鲁棒性。为验证该方法的有效性,分别在FB15K-237和WN18RR数据集上进行了实验,并将实验结果与RotatE、IMF和MLSFF等14种方法进行对比分析,结果表明了所提方法的有效性。

     

    Abstract: To address the issues of visual modality information loss and multimodal noise in multimodal knowledge reasoning tasks, a knowledge reasoning method based on adversarial networks and relation-guided attention is proposed. Through multimodal adversarial learning, the existing textual modality information is utilized to generate the missing visual modality features, thereby alleviating the information imbalance and entity alignment problems caused by modality loss. Meanwhile, a relation-guided attention mechanism is designed to capture important multimodal features relevant to relation context, thereby reducing multimodal noise to mitigate the problems of misalignment and feature offset duo to incomplete information, feature sparsity, and cross-modal semantic inconsistency. To validate this method, experiments are conducted on the FB15K-237 and WN18RR datasets, comparing the results with 14 methods including RotatE, IMF, and MLSFF, demonstrating the effectiveness of this approach.

     

/

返回文章
返回