基于Self-Attention和Bi-LSTM的中文短文本情感分析

Abstract
Figure/Table
References
Related Citation (15)

Download: PDF (2207 KB) HTML (1 KB)
Export: BibTeX | EndNote (RIS)

Abstract Short text sentiment analysis is a better method for judging the emotions of texts. It also has important applications in the fields of commodity reviews and public opinion monitor. The performance of the bidirectional recurrent neural network model based on the word attention mechanism relies heavily on the accuracy of word segmentation. In addition, the attention mechanism has more parameter dependencies, making the model less concerned with the internal sequence relationships of short texts. Aiming at the above problems, this paper proposes a Chinese short text sentiment analysis algorithm based on the character vector representation method combined with Self-attention and BiLSTM. Firstly, the short text is vectrized, then the BiLSTM network is used to extract texts context feature. Finally, the feature weights are dynamically adjusted by the self-attention mechanism, and the Softmax classifier obtains the emotion category. Experimental results on the COAE 2014 Weibo dataset and hotel review datasets show that character vectors are more suitable for short text than word-level text vector representations. The self-attention mechanism can reduce the external parameter dependence, so that the model can learn more key features of the text itself. Classification performance can be increased by 1.15% and 1.41%, respectively.

Key words： sentiment analysis character vector self-attention BiLSTM

Received: 17 July 2018

ZTFLH:

TP391

	Service

	E-mail this article
	Add to my bookshelf
	Add to citation manager
	E-mail Alert
	RSS
	Articles by authors
	WU Xiaohua
	CHEN Li
	WEI Tiantian
	FAN Tingting

Cite this article:

WU Xiaohua,CHEN Li,WEI Tiantian, et al. Sentiment Analysis of Chinese Short Text Based on Self-Attention and Bi-LSTM[J]. , 2019, 33(6): 100-107.

URL:

http://jcip.cipsc.org.cn/EN/ OR http://jcip.cipsc.org.cn/EN/Y2019/V33/I6/100

[1] 王仲远,程建鹏,王浩勋,等.短文本理解研究[J].计算机研究与发展,2016,53 (2):262-269.
[2] Picard R W.Affective computing:challenges[J].International Journal of Human-Computer Studies,2003,59(s1-2):55-64.
[3] Jeffrey L Elman.Finding structure in time[J].Cognitive Science.1990,14(2):179-211.
[4] Hochreiter S,Schmidhuber J.Long short-term memory[J].Neural Computation,1997,9(8):1735-1780.
[5] Esin Y E,Alan O,Alpaslan F N.Improvement on corpus-based word similarity using vector space models[C]//Proceedings of the International Symposium on Computer and Information Sciences.IEEE,2009:280-285.
[6] Blei D M,Ng A Y,Jordan M I.Latent dirichlet allocation[J].Journal of Machine Learning Research Archive,2003,3:993-1022.
[7] Bengio Y,Ducharme R,Vincent P,et al.A neural probabilistic language model[C]//Proceedings of the Innovations in Machine Learning.Springer Berlin Heidelberg,2006:137-186.
[8] Mikolov T,Corrado G,Chen K,et al.Efficient estimation of word representations in vector space[C]//Proceedings of the International Conference on Learning Representations.2013:1-12.
[9] 许云,樊孝忠,张锋.一种不需分词的中文文本分类方法[J].北京理工大学学报,2015,25(9):778-781.
[10] 刘龙飞,杨亮,张绍武,等.基于卷积神经网络的微博情感倾向性分析[J].中文信息学报,2015,29(6):159-165.
[11] Mnih V,Heess N,Graves A.Recurrent models of visual attention[C]//Proceedings of Advances in neural information processing systems.2014:2204-2212.
[12] Bahdanau D,Cho K,Bengio Y.Neural machine translation by jointly learning to align and translate[J].arXiv preprint arXiv:1409.0473,2016:1-15.
[13] Vaswani,Ashish,et al.Attention is all you need[J].Advances in Neural Information Processing Systems.2017,22(7):139-147.
[14] Pavlopoulos J,Malakasiotis P,Androutsopoulos I.Deeper attention to abusive user content moderation[C]//Proceedings of the Conference on Empirical Methods in Natural Language Processing.2017:1125-1135.
[15] Zhao Q L,Cai X D,et al.Text feature extraction method based on LSTM-Attention neural network[J].Modern Electronics Technique.2018:41(8):167-170.
[16] 江伟,金忠.基于短语注意机制的文本分类[J].中文信息学报,2018,32(2):102-109.
[17] Yao X.Attention-based BiLSTM neural networks for sentiment classification of short texts[J].Proceedings of Science.2017:1-8.
[18] Cheong M,Lee V.Dissecting Twitter:A review on current microblog research and lessons from related fields[M].From Sociology to Computing in Social Networks.Springer,Vienna,2010:343-362.
[19] Kim Y,Denton C,Hoang L,Alexander M.Rush.Structured attention networks[C]//Proceedings of International Conference on Learning Representations,2017:1-21.
[20] Chen Y,Zhao X,Jia X.Spectral—spatial classification of hyperspectral data based on deep belief network[J].IEEE Journal of Selected Topics in Applied Earth Observations & Remote Sensing,2015,8 (6):2381-2392.
[21] Wojciech Zaremba,Ilya Sutskever,Oriol Vinyals.Recurrent neural network regularization[J].Eprint Arxiv,2014:1049-1329.
[22] Wang Y Q,Huang M,Zhao L,et al.Attention-based LSTM for aspect-level sentiment classification[C]//Proceedings of Conference on Empirical Methods in Natural Language Processing.2016:606-615.
[23] 陈钊,徐睿峰,桂林,等.结合卷积神经网络和词语情感序列特征的中文情感分析[J].中文信息学报,2015,29(6):172-178.
[24] 冯兴杰,张志伟,史金钏.基于卷积神经网络和注意力模型的文本情感分析[J].计算机应用研究,2018(5):1-4.