Multimodal Emotion Recognition in Conversation Based on Cauchy Kernel Granulation Rough Set Transformer

Transformer is widely used in multimodal conversation sentiment analysis tasks. However, for inconsistent expressions in the data, existing attention mechanisms directly multiply uncertain features for feature guidance, which ultimately leads to confusion in the guidance of the Transformer. To address the uncertainty problem, rough sets provide tolerance by measuring the relationships between features from a set-theoretic perspective, making them an effective mathematical tool for handling uncertain features. To alleviate the uncertain features, this paper proposes an attention mechanism based on Cauchy kernel granulation rough set. Firstly, the Cauchy kernel function is used to granulate the features to form an information granule, and then the lower approximation is performed on the information granule for feature guidance. Finally, the constructed Cauchy kernel granulation rough set attention is integrated into the Transformer network for the multimodal conversation sentiment analysis task. Experiments and analysis on the IEMOCAP, CMU-MOSEI and MELD datasets show that the proposed method outperforms most of the state-of-the-art models, ranking top in on the IEMOCAP dataset with 84.71% W_F₁ score, achieving current state-of-the-art performance.