当前存在着大量的语言知识和情感资源,但在基于深度学习的情感分析研究中,这些特有的情感信息,没有在情感分析任务中得到充分利用。针对以上问题,该文提出了一种基于多通道双向长短期记忆网络的情感分析模型(multi-channels bidirectional long short term memory network,Multi-Bi-LSTM),该模型对情感分析任务中现有的语言知识和情感资源进行建模,生成不同的特征通道,让模型充分学习句子中的情感信息。与CNN相比,该模型使用的Bi-LSTM考虑了词序列之间依赖关系,能够捕捉句子的上下文语义信息,使模型获得更多的情感信息。最后在中文COAE2014数据集、英文MR数据集和SST数据集进行实验,取得了比普通Bi-LSTM、结合情感序列特征的卷积神经网络以及传统分类器更好的性能。
Abstract
Language knowledge and sentiment resources are not well utilized in the current deep learning sentiment analysis. To address this issue, we propose a novel sentiment analysis model based on multi-channel bidirectional long short term memory network (Multi-Bi-LSTM), which generates different feature channels to fully learn the sentiment information of the text. Compared with CNN, the Bi-LSTM used in this model takes into account the dependencies between word sequences, and it can capture contextual semantic information about a sentence. The experiments on Chinese COAE2014 dataset, English MR dataset and SST dataset reveal better performance of the proposed method than the classical Bi-LSTM, the CNN combined with the features of sentiment sequences, and the classical classifiers.
关键词
情感分析 /
长短期记忆 /
多通道 /
层归一化
{{custom_keyword}} /
Key words
sentiment analysis /
long short term memory /
multi-channels /
layer normalization
{{custom_keyword}} /
{{custom_sec.title}}
{{custom_sec.title}}
{{custom_sec.content}}
参考文献
[1] Nasukawa T, Yi J. Sentiment analysis: Capturing favorability using natural language processing[C]//Proceedings of the International Conference on Knowledge Capture. New York: ACM, 2003: 70-77.
[2] Boiy E, Moens M F. A machine learning approach to sentiment analysis in multilingual Web texts.[J]. Information Retrieval. 2009, 12(5): 526-558.
[3] Ye Q, Zhang Z Q, Law R. Sentiment classification of online reviews to travel destinations by supervised machine learning approaches[J]. Expert Systems with Applications. 2009, 36(3): 6527-6535.
[4] Kim Y. Convolutional neural networks for sentence classification[C]//Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing. Stroudsburg, PA: ACL, 2014: 1746-1751.
[5] Mikolov T, Karafiát M, Burget L, et al. Recurrent neural network based language model[C]//Proceedings of the 11th Annual Conference of the International Speech Communication Association. Japan: 2010: 1045-1048.
[6] Zhu X D, Sobihani P, Guo Hongyu. Long short-term memory over recursive structures[C]//Proceedings of the International Conference on Machine Learning. New York: ACM, 2015: 1604-1612.
[7] 陈钊,徐睿峰,桂林,等. 结合卷积神经网络和词语情感序列特征的中文情感分析[J]. 中文信息学报. 2015, 29(6): 172-178.
[8] Qian Q, Huang M, Lei J, et al. Linguistically regularized LSTMs for sentiment classification[C]//Proceedings of the ACL. 2017: 1679-1689.
[9] Tai K S, Socher R, Manning C D. Improved semantic representations from tree-structured long short-term memory Networks[C]//Proceedings of the ACL. 2015: 1556-1566.
[10] 黄发良,冯时,王大玲,等. 基于多特征融合的微博主题情感挖掘[J]. 计算机学报. 2017, 40(04): 102-118.
[11] 刘龙飞,杨亮,张绍武,等. 基于卷积神经网络的微博情感倾向性分析[J]. 中文信息学报. 2015, 29(6): 159-165.
[12] Socher R, Pennington J, Huang E H, et al. Semi-supervised recursive autoencoders for predicting sentiment distributions[C]//Proceedings of the Empirical Methods in Natural Language Processing. Cambridge, MA: MIT Press, 2011: 151-161.
[13] Socher R, Huval B, Manning C D, et al. Semantic compositionality through recursive matrix-vector spaces[C]//Proceedings of the 2012 Joint Conference on Empirical Methods in Natural Language Processing and Computational Natural Language Learning. Cambridge, MA: MIT Press, 2012: 1201-1211.
[14] Socher R, Perelygin A, Wu J Y, et al. Recursive deep models for semantic compositionality over a sentiment Treebank[C]//Proceedings of the Empirical Methods in Natural Language Processing. Cambridge, MA: MIT Press, 2013: 1631-1642.
[15] Wang J, Yu L C, Lai K R, et al. Dimensional sentiment analysis using a regional CNN-LSTM model[C]//Meeting of the Association for Computational Linguistics. 2016: 225-230.
[16] Liu Q, Zhang H B, et al. Content Attention Model for Aspect Based Sentiment Analysis[C]//Proceedings of the In WWW 2018: The 2018 Web Conference. Lyon, France: 2018: 1023-1032.
[17] Nallapati R, Zhou B, Santos C N D, et al. Abstractive text summarization using sequence-to-sequence RNNs and beyond[J]. arXiv preprint arXiv: 1602.06023,2016.
[18] Schuster M, Paliwal K K. Bidirectional recurrent neural networks[J]. IEEE Transactions on Signal Processing. 2002, 45(11): 2673-2681.
[19] Ba J L, Kiros J R, Hinton G E. Layer Normalization[J]. arXiv preprint arXiv: 1607.06450,2016.
[20] Manning C D. Accurate unlexicalized parsing[C]//Proceedings of Meeting of the Association for Computational Linguistics. 2003: 423-430.
[21] Pennington J, Socher R, Manning C D. Glove Global vectors for word representation[C]//Proceedings of the International Conference on Empirical Methods in Natural Language Processing. ACL: 2014: 1532-1543.
[22] Duchi J, Hazan E, Singer Y. Adaptive Subgradient Methods for Online Learning and Stochastic Optimization[J]. Journal of Machine Learning Research. 2011, 12(7): 257-269.
[23] Lei T, Barzilay R, Jaakkola T.Molding CNNs for text: non-linear, non-consecutive convolutions [J]. arXiv preprint arXiv: 1508.04112,2015.
{{custom_fnGroup.title_cn}}
脚注
{{custom_fn.content}}
基金
国家自然科学基金(61363045)
{{custom_fund}}
PDF(2261 KB)
