融合卷积神经网络与双向GRU的文本情感分析胶囊模型

程艳,孙欢,陈豪迈,李猛,蔡盈盈,蔡壮

PDF(3354 KB)
PDF(3354 KB)
中文信息学报 ›› 2021, Vol. 35 ›› Issue (5) : 118-129.
情感分析与社会计算

融合卷积神经网络与双向GRU的文本情感分析胶囊模型

  • 程艳1,孙欢1,陈豪迈2,李猛1,蔡盈盈1,蔡壮1
作者信息 +

Text Sentiment Analysis Capsule Model Combining Convolutional Neural Network and Bidirectional GRU

  • CHENG Yan1, SUN Huan1, CHEN Haomai2, LI Meng1, CAI Yingying1, CAI Zhuang1
Author information +
History +

摘要

文本情感分析是自然语言处理领域一个重要的分支。现有深度学习方法不能更为全面地提取文本情感特征,且严重依赖于大量的语言知识和情感资源,需要将这些特有的情感信息充分利用使模型达到最佳性能。该文提出了一种融合卷积神经网络与双向GRU网络的文本情感分析胶囊模型。该模型首先使用多头注意力学习单词间的依赖关系、捕获文本中情感词,利用卷积神经网络和双向GRU提取文本不同粒度的情感特征,特征融合后输入全局平均池化层,在得到文本的实例特征表示的同时,针对每个情感类别结合注意力机制生成特征向量构建情感胶囊,最后根据胶囊属性判断文本情感类别。模型在MR、IMDB、SST-5及谭松波酒店评论数据集上进行实验,相比于其他基线模型具有更好的分类效果。

Abstract

Text sentiment analysis is an important branch in the field of natural language processing. This paper proposes a text sentiment analysis capsule model that combines convolutional neural networks and bidirectional GRU networks. Firstly, the multi-head attention is used to learn the dependency between words and capture the emotional words in the text. Then, the convolutional neural network and bidirectional GRU network are used to extract emotional features of different granularities in the text. After the feature fusion, the global average pooling is used to get the instance feature representation of the text, and the attention mechanism is combined to generate feature vectors for each emotion category to construct an emotion capsule. Finally, the emotion category of the text is judged by the capsule attributes. Tested on the MR, IMDB, SST-5 and Tan Songbo hotel review datasets, the proposed model achieves better classification effect than other baseline models.

关键词

文本情感分析 / 多头注意力 / 卷积神经网络 / 双向门控循环网络 / 情感胶囊

Key words

text sentiment analysis / multi-head attention / convolutional neural network / bidirectional gated recurrent unit network / sentiment capsule

引用本文

EndNote

Ris (Procite)

Bibtex

导出引用
程艳,孙欢,陈豪迈,李猛,蔡盈盈,蔡壮. 融合卷积神经网络与双向GRU的文本情感分析胶囊模型. 中文信息学报. 2021, 35(5): 118-129
CHENG Yan, SUN Huan, CHEN Haomai, LI Meng, CAI Yingying, CAI Zhuang. Text Sentiment Analysis Capsule Model Combining Convolutional Neural Network and Bidirectional GRU. Journal of Chinese Information Processing. 2021, 35(5): 118-129

参考文献

[1] 李然,林政,林海伦,等.文本情绪分析综述[J].计算机研究与发展,2018,55(01): 30-52.
[2] Liu B. Sentiment analysis: Mining opinions, sentiments, and emotions[J]. Computational Linguistics, 2015, 42(3): 1-4.
[3] Nasukawa T, Yi J. Sentiment analysis: Capturing favorability using natural language processing[C]//Proceedings of the 2nd International Conference on Knowledge Capture, 2003: 70-77.
[4] Xiao Z, Li X, Wang L, et al. Using convolution control block forChinese sentiment analysis[J]. Journal of Parallel and Distributed Computing, 2018,116: 18-26.
[5] Zhang Y, Jin R, Zhou Z H. Understanding bag-of-words model: A statistical framework[J]. International Journal of Machine Learning and Cybernetics, 2010, 1(1-4): 43-52.
[6] 陈钊,徐睿峰,桂林,等.结合卷积神经网络和词语情感序列特征的中文情感分析[J].中文信息学报,2015,29(06): 172-178.
[7] 刘龙飞,杨亮,张绍武,等.基于卷积神经网络的微博情感倾向性分析[J].中文信息学报, 2015, 29(6): 159-165.
[8] Qian Q, Huang M, Lei J, et al. Linguistically regularized LSTMs for sentiment classification[J]. Computational Linguistics,2016,14(4): 34-37.
[9] Teng Z, Vo D T, Zhang Y. Context-sensitive lexicon features for neural sentiment analysis[C]// Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing. 2016: 1629-1638.
[10] 陈珂,梁斌,柯文德,等.基于多通道卷积神经网络的中文微博情感分析[J].计算机研究与发展,2018, 55(005): 945-957.
[11] 李卫疆,漆芳.基于多通道双向长短期记忆网络的情感分析[J].中文信息学报,2019,33(12): 119-128.
[12] Wang Y, Sun A, Han J, et al. Sentiment analysis by capsules[C]//Proceedings of the Web Conference. 2018: 1165-1174.
[13] Hinton G E,Krizhevsky A, Wang S D. Transforming auto-encoders[C]//Proceeding of the 21st International Conference on Artificial Neural Networks. Espoo, Finland, Springer, 2011: 44-51.
[14] Sabour S, Frosst N, Hinton G. Dynamic routing between capsules[C]//Proceedings of Advances in Neural Information processing systems, 2017: 3856-3866.
[15] Kim J, Jang S, Choi S, et al. Text classification using capsules[J].Neurocomputing, 2018,376: 214-221.
[16] Zhao W, Ye J, Yang M, et al. Investigating capsule networks with dynamic routing for textclassification[J]. arXiv preprint arXiv: 1804.00538, 2018.
[17] 贾旭东,王莉.基于多头注意力胶囊网络的文本分类模型[J].清华大学学报(自然科学版),2020,60(05): 415-421.
[18] Zhao Z, Wu Y. Attention-based convolutional neural networks for sentenceclassification[C]//Proceeding of the 2016 Annual Conference of the International Speech Communication Association. Baixas: ISCA, 2016: 705-709.
[19] Vaswani A, Shazeer N, Parmar N, et al. Attention is all you need[J]. arXiv preprint arXiv: 1706.03762, 2017.
[20] Lin M, Chen Q, Yan S,et al. Network in network[J]. arXiv preprint arXiv: 1312.4400,2013.
[21] Pang B, Lillian L,Vaithyanathan S. Thumbs up?: Sentiment classification using machine learning techniques[C]//Proceedings of the ACL-02 Conference on Empirical Methods in Natural Language Processing. Stroudsburg, PA: ACL, 2002: 79-86.
[22] 唐庄,王志舒,周爱,等.面向文本分类的transformer-capsule集成模型[J].计算机工程与应用,2020,56(24): 151-156.
[23] Kim Y. Convolutional neural networks for sentenceclassification[J]. arXiv preprint arXiv: 1408.5882,2014.
[24] Zhang X, Zhao J, Lecun Y. Character-level convolutional networks for text classification[C]//Proceedings of Advances in Neural Information Processing Systems,2015: 645-657.
[25] 程艳,叶子铭,王明文,等.融合卷积神经网络与层次化注意力网络的中文文本情感倾向性分析[J].中文信息学报,2019,33(01): 133-142.
[26] Socher R, Perelygin A, Wu J, et al. Recursive deep models for semantic compositionality over a sentiment treebank[C]//Proceedings of the Empirical Methods in Natural Language Processing. Cambridge, MA: MIT Press,2013: 1631-1642.
[27] Tai K S,Socher R, Manning C D. Improved semantic representations from tree-structured long short-term memory networks[C]//Proceeding of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing. 2015: 1556-1566.
[28] Cho K, Merrienboer B V, Gulcehre C, et al. Learning phrase representations using RNN encoder-decoder for statistical machine translation[J]. arXiv preprint arXiv: 1406.1078,2014.
[29] Zhang H, Jin W, Zhang J, et al. Ynu-hpcc at semeval 2017 task 4: Using a multi-channel CNN-LSTM model for sentiment classification[C]//Proceeding of the 11th International Workshop on Semantic Evaluation. Stroudsburg, PA: ACL, 2017: 796-801.
[30] 袁和金,张旭,牛为华,等.融合注意力机制的多通道卷积与双向GRU模型的文本情感分析研究[J].中文信息学报,2019,33(10): 109-118.
[31] Rehman A U, Malik A K, Raza B, et al. A hybrid CNN-LSTM model for improving accuracy of movie reviews sentiment analysis[J]. Multimedia Tools and Applications 2019,78(18): 26597-26613.
[32] Katarya R, Arora Y. Study on text classification using capsule networks[C]//Proceedings of the 5th International Conference on Advanced Computing and Communication Systems, 2019: 501-504.
[33] 朱张,饶元,吴渊,等.注意力机制在深度学习中的研究进展[J].中文信息学报,2019,33(06): 1-11.
[34] Lin Z, Feng M, Santos C N D, et al. A structured self-attentive sentence embedding[J]. arXiv: 1703.03130, 2017.
[35] Bahdanau D, Cho K, Bengio Y. Neural machine translation by jointly learning to align and translate[J]. arXiv preprint arXiv: 1409.0473, 2014.
[36] Pang B, Lee L. Seeing stars: Exploiting class relationships for sentiment categorization with respect to rating scales[C]//Proceedings of the 43rd Annual Meeting on Association for Computational Linguistic. Ann Arbor, ichigan, SA, 2005: 115-124.
[37] Maas A, Daly R E, Pham P T, et al. Learning word vectors for sentiment analysis[C]//Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies. 2011: 142-150.
[38] Huang W, Cheng X, Chen K, et al. Toward fast and accurate neural Chinese word segmentation with multi-criteria learning[J]. arXiv preprint arXiv: 1903.04190, 2019.
[39] Wang S, Manning C D. Baselines and bigrams: Simple, good sentiment and topic classification[C]// Proceedings of the 50th Annual Meeting of the Association for Computational Linguistics: Short Papers-Vol 2. Association for Computational Linguistics, 2012: 90-94.
[40] Devlin J, Chang M W, Lee K, et al. BERT: Pre-training of deep bidirectional transformers for language understanding[C]//Proceedings of the 2019 conference of the North American Chapter of the Association for Computational Linguistics. Stroudsburg, PA: ACL, 2019: 4171-4186.
[41] Sun X, Gao Y, Sutcliffe R, et al. Word representation learning based on bidirectional GRUs with drop loss for sentiment classification[J]. IEEE Transactions on Systems, Man, and Cybernetics: Systems, 2019, 99: 1-11.
[42] 杨晨,宋晓宁,宋威. SentiBERT: 结合情感信息的预训练语言模型[J].计算机科学与探索,2020,14(09): 1563-1570.
[43] Yu J, Jiang J. Learning sentence embeddings with auxiliary tasks for cross-domain sentiment classification[C]//Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing. Stroudsburg: ACL, 2016: 236-246.

基金

国家自然科学基金(61967011);江西省自然科学基金(20202BABL202033);江西省科技攻关重点研发项目(20161BBE50086);江西省教育厅科技重点项目(GJJ150299);江西省教育厅人文社科重点项目(JD19056);国家社会科学基金(20AXW009)
PDF(3354 KB)

Accesses

Citation

Detail
段落导航
相关文章

  • 版权所有 © 《中文信息学报》编辑部 网站统计

    地址：北京市海淀区中关村南四街4号　邮编：100190　电话：010-62562916　E-mail：cips@iscas.ac.cn

    本系统由北京玛格泰克科技发展有限公司设计开发

/

〈 〉