Helical Attention Networks for Aspect-level Sentiment Classification
DU Chengyu1, LIU Pengyuan2
1.School of Information Science, Beijing Language and Culture University, Beijing 100083, China; 2.Language Resources Monitoring &Research Center, Beijing Language and Culture University, Beijing 100083, China
Abstract：Aspect-level sentiment classification is a fine-grained sentiment analysis task, with the purpose to identify the sentiment polarity for a particular aspect. This paper proposes a BERT-based Helical Attention Networks (BHAN) which employ a helical attention mechanism to get a better representation of context and aspect. Specifically, on the basis of the weighted context representation based on averaged aspect vector, we use it to compute the attention weight of aspect. Then we use the new weight aspect representation to compute the context attention weight again. We can get a better representation of context and aspect by iterate above process until convergence. Evaluated on SemEval 2014 Task 4 and Twitter dataset, the proposed method out-performs the existing state-of-the-art methods.
[1] Svetlana Kiritchenko, Xiaodan Zhu, Colin Cherry,et al. Nrc-canada-2014: Detecting aspects and sentiment in customer reviews[C]//Proceedings of SemEval, 2014: 437-442. [2] Joachim Wagner, Piyush Arora, Santiago Cortes, et al. Dcu: Aspect-based polarity classification for sem eval task 4[C]//Proceedings of SemEval, 2014: 223-229. [3] Duy-Tin Vo, Yue Zhang. Target-dependent twitter sentiment classification with rich automatic features[C]//Proceedings of IJCAI, 2015: 1347-1353. [4] Richard Socher, Jeffrey Pennington, Eric H. Huang, et al. Semi-supervised recursive autoencoders for predicting sentiment distributions[C]//Proceedings of the 2011 Conference on Empirical Methods in Natural Language Processing, 2011: 151-161. [5] Richard Socher, Alex Perelygin, Jean Wu, et al. Recursive deep models for semantic compositionality over a sentiment treebank[C]//Proceedings of the 2013 Conference on Empirical Methods in Natural Language Processing, 2013. [6] Kai Sheng Tai, Richard Socher, et al. Improved semantic representations from tree-structured long short-term memory networks[C]//Proceedings of ACL-2015, 2015: 1556-1566. [7] Tang Duyu, Qin Bing, et al. Effective LSTMs for target-dependent sentiment classification[C]//Proceed-ings of COLING-2016, 2016(a): 3298-3307. [8] Wang Yequan, Huang Minlie, et al. Attention-based LSTM for aspect-level sentiment classification[C]//Proceedings of EMNLP-2016, 2016: 606-615. [9] Tang Duyu, Qin Bing, et al. Aspect level sentiment classification with deep memory network[C]//Proceedings of EMNLP-2016, 2016(b): 214-224. [10] Ma Dehong, Li Sujian, Zhang Xiaodong, et al. Interactive attention networks for aspect-level sentiment classification[C]//Proceedings of IJCAI-2017, 2017: 4068-4074. [11] Ding Xiaowen, Liu Bing, Philip S Yu. A holistic lexicon-based approach to opinion mining[C]//Proceedings of the 2008 International Conference on Web Search and Data Mining, 2008: 231-240. [12] Jiang Long, Yu Mo, Zhou Ming, et al. Target-dependent twitter sentiment classification[C]//Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies-Volume 1, 2011: 151-160. [13] Zhao Wayne Xin, Jiang Jing, Yan Hongfei, et al. Jointly modeling aspects and opinions with a maxent-lda hybrid[C]//Proceedings of the 2010 Conference on Empirical Methods in Natural Language Processing, EMNLP’10, 2010: 56-65. [14] Chen P, Sun Z, Bing L, et al. Recurrent attention network on memory for aspect sentiment analysis[C]//Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, 2017: 452-461. [15] Zheng Shiliang, Xia Rui. Left-center-right separated neural network for aspect-based sentiment analysis with rotatory attention[J]. arXiv preprint arXiv: 1802.00892, 2018. [16] Huang Binxuan, Ou Yanglan, Kathleen M Carley. Aspect level sentiment classification with attention-over-attention neural networks[J]. arXiv preprint ar Xiv: 1804.06536, 2018. [17] Li L, Liu Y, Zhou A Q. Hierarchical attention based position-aware network for aspect-level sentiment analysis[C]//Proceedings of the 22nd Conference on Computational Natural Language Learning, 2018: 181-189. [18] Song Y, Wang J, Jiang T, et al. Attentional encoder network for targeted sentiment classification[J]. arXiv preprint arXiv: 1902.09314, 2019. [19] Zhao Pinlong, Hou Linli, Wu Ou. Modeling sentiment dependencies with graph convolutional networks for aspect-level sentiment classification[J]. arXiv preprint arXiv: 1906.04501, 2019. [20] Devlin J, Chang M W, Lee K, et al. Bert: Pre-training of deep bidirectional transformers for language understanding[J]. arXiv preprint arXiv: 1810.04805, 2018. [21] Schuster M, Paliwal K. Bidirectional recurrent neural networks[M]. NY: IEEE Press, 1997. [22] Joachim Wagner, Piyush Arora, Santiago Cortes, et al. Dcu: Aspect-based polarity classification for semeval task 4[C]//Proceedings of Sem Eval, 2014: 223-229. [23] Dong Li, Wei Furu, Tan Chuanqi, et al. Adaptive recursive neural network for target-dependent twitter sentiment classification[C]//Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics, 2014: 49-54. [24] Diederik P Kingma, Jimmy Ba. Adam: A method for stochastic optimization[J]. arXiv preprint arXiv: 1412.6980. 2014. [25] Jeffrey Pennington, Richard Socher, Christopher Manning. Glove: Global vectors for word representation[C]//Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP), 2014: 1532-1543.
2020, Vol. 34 
