高级检索

HMUM:面向仇恨模因检测的多阶段多模态理解模型

HMUM: A Multi-stage Multimodal Understanding Model for Hateful Meme Detection

  • 摘要: 针对现有仇恨模因(meme)检测方法对语义隐晦、文化特定内容识别效果有限的问题,该文提出基于视觉大模型Qwen2.5-VL-72B-Instruct的仇恨模因理解模型(Hateful Meme Understanding Model, HMUM)。该方法采用LoRA微调技术,构建多阶段提示学习框架,通过文本识别、情绪建模与仇恨推理的渐进分析流程提升检测性能。在ToxiCN MM数据集上的评估结果显示,HMUM在整体仇恨模因检测任务中取得81.7%的F1值,较基线模型MKE提升8.4%;在隐性仇恨子集上的召回率达到70.6%,F1值提升至82.8%。基于自建数据集ITTD-220的进一步验证表明,模型取得73.5%的召回率和83.6%的F1值,证实了该方法在不同测试环境下的稳定性和泛化能力。

     

    Abstract: Current detection methods are less effective for hateful meme with implicit semantics and culture-specific content. This study developed HMUM, a hateful meme understanding model based on Qwen2.5-VL-72B-Instruct. The model employed LoRA fine-tuning and a multi-stage prompt learning framework. It enhanced detection performance through progressive analysis of text recognition, emotion modeling, and hate reasoning. Evaluation on the ToxiCN MM dataset showed that HMUM achieved an 81.7% F1-score for overall hate meme detection, i.e. an 8.4% improvement over the MKE baseline. For the implicit hate meme subset, the model reached 70.6% recall and 82.8% F1-score.

     

/

返回文章
返回