基于多任务预训练模型的属性级情感分类

周敏,王中卿,李寿山,周国栋

PDF(5123 KB)
PDF(5123 KB)
中文信息学报 ›› 2022, Vol. 36 ›› Issue (10) : 126-134.
情感分析与社会计算

基于多任务预训练模型的属性级情感分类

  • 周敏,王中卿,李寿山,周国栋
作者信息 +

Aspect-level Sentiment Classification Based on Multi-Task Pre-Training Model

  • ZHOU Min, WANG Zhongqing, LI Shoushan, ZHOU Guodang
Author information +
History +

摘要

目前,缺少标注样本数据是属性级情感分类任务面临的一大难题,为了解决这一问题,该文提出了结合多项任务的预训练Bert模型。该模型利用大量未标注的篇章级情感分类数据,结合多种分类任务预训练模型共享参数,迁移属性级评论和篇章级评论中共享的有用的语义语法信息,从而帮助模型提高属性级情感分类准确率。在SemEval-14数据集上的实验结果表明,相较于一系列基准模型,该文提出的模型有效提高了属性级情感分类的准确率。

Abstract

The lack of annotated sample has become a major challenge for aspect-level sentiment classification. This paper proposes a combined multi-task pre-training Bert model to alleviate this issue. A large amount of unlabeled document-level sentiment classification data is employed to train a variety of classification tasks for a pre-trained model with share parameters, so as to transfer the useful semantic and grammatical information shared between aspect-level comments and document-level comments. Experiments on the SemEval-14 data set show that, compared with a series of baseline models, the model proposed in this paper effectively improves the accuracy of aspect-level sentiment classification.

关键词

Bert / 多任务 / 情感分类

Key words

Bert / multi-masks / sentiment classification

引用本文

导出引用
周敏,王中卿,李寿山,周国栋. 基于多任务预训练模型的属性级情感分类. 中文信息学报. 2022, 36(10): 126-134
ZHOU Min, WANG Zhongqing, LI Shoushan, ZHOU Guodang. Aspect-level Sentiment Classification Based on Multi-Task Pre-Training Model. Journal of Chinese Information Processing. 2022, 36(10): 126-134

参考文献

[1] Pang B, Lee L. Opinion mining and sentiment analysis[J]. Foundations and Trends in Information Retrieval, 2008, 2(1): 1-135.
[2] Jo Y, Oh A. Aspect and sentiment unification model for online review analysis[C]//Proceedings of the 4th ACM International Conference on Web Search And Data Mining, 2011: 815-824.
[3] Pontiki M, Galanis D, Papageorgiou H, et al. SemEval-2016 task 5: Aspect based sentiment analysis[C] //Proceedings of the 10th International Workshop on Semantic Evaluation, 2016: 19-30.
[4] Jiang L, Yu M, Zhou M, et al. Target-dependent Twitter sentiment classification[C]//Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics, 2011: 151-160.
[5] Mohammad S M,Kiritchenko S, Zhu X, et al. NRC-Canada: Building the state-of-the-art in sentiment analysis of tweets[C] //Proceedings of the 2nd Joint Conference on Lexical and Computational Semantics, 2013: 321-327.
[6] Wagner J, Arora P, Cortes S, et al. DCU: Aspect-based polarity classification for semEval Task 4[C]//Proceedings of the 8th International Workshop on Semantic Evaluation, 2014: 223-229.
[7] Le C Y, Yoshua B, Geoffrey H. Deep learning[J]. Nature 521(7553):436-444.
[8] Dong L, Wei F, Tan C, et al. Adaptive recursive neural network for target-dependent twitter sentiment classification[C]//Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics. 2014: 49-54.
[9] Himabindu L, Richard S, Chris M. Aspect specific sentiment analysis using hierarchical deep learning[C]//Proceedings of the NIPS Workshop on Deep Learning and Representation Learning,2014.
[10] Vo D T, Zhang Y. Target-dependent twitter sentiment classification with rich automatic features[C]//Proceedings of the 24th International Conference on Artificial Intelligence International Conference on Artificial Intelligence, 2015: 1347-1353.
[11] Hochreiter S, Schmidhuber J. Long short-term memory[J]. Neural Computation, 1997, 9(8): 1735-1780.
[12] Le C Y, Bottou L, Bengio Y, et al. Gradient based learning applied to document recognition[J]. Proceedings of the IEEE, 1998, 86(11): 2278-2324.
[13] Tang D, Qin B, Feng X, et al. Effective LSTMs for target-dependent sentiment classification[C]//Proceedings of COLING, the 26th International Conference on Computational Linguistics: Technical Papers, 2016: 3298-3307.
[14] Elman J L. Finding structure in time[J]. Cognitive Science, 1990, 14(2): 179-211.
[15] Wang Y, Huang M, Zhu X, et al. Attention-based LSTM for aspect-level sentiment classification[C]//Proceedings of the Conference on Empirical Methods in Natural Language Processing, 2016: 606-615.
[16] Yang M, Tu W, Wang J, et al. Attention based LSTM for target dependent sentiment classification. [C]//Proceedings of the 31st AAAI Conference on Artificial Intelligence, 2017: 5013-5014.
[17] Huang B, Carley K M. Parameterized convolutional neural networks for aspect level sentiment classification[C]//Proceedings of the Conference on Empirical Methods in Natural Language Processing,2018: 1091-1096.
[18] Xue W, Li T. Aspect based sentiment analysis with gated convolutional networks[C]//Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics, 2018: 2514-2523.
[19] Li X, Bing L, Lam W, et al. Transformation networks for target-oriented sentiment classification[C]//Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics,2018: 946-956.
[20] Devlin J, Chang M, Lee K, et al. BERT: Pre-training of deep bidirectional transformers for language understanding[C]//Proceedings of the Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, 2019: 4171-4186.
[21] Sun C, Huang L,Qiu X, et al. Utilizing BERT for aspect-based sentiment analysis via constructing auxiliary sentence. [C]//Proceedings of North American Chapter of the Association for Computational Linguistics, 2019: 380-385.
[22] Xu H, Liu B, Shu L, et al. BERT post-training for review reading comprehension and aspect-based sentiment analysis[J].arXiv: Computation and Language, 2019.
[23] Pontiki M, Galanis D, Pavlopoulos J, et al. SemEval-2014 Task 4: Aspect based sentiment analysis[C]//Proceedings of the 8th International Workshop on Semantic Evaluation, 2014: 27-35.
[24] Peters M E, Neumann M,Iyyer M, et al. Deep contextualized word representations [C]// Proceedings of the Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, 2018: 2227-2237.
[25] Radford Alec, Narasimhan Karthik,Salimans Tim, et al. Improving language understanding by generative pre-training[EB/OL]https://openai.com/blog/language-unsupervised/[2019-08-29].
[26] Wu Y, Schuster M, Chen Z, et al. Google's neural machine translation system: Bridging the gap between human and machine translation[J].arXiv preprint arXiv:1609.08144, 2016.
[27] Tang D, Qin B, Liu T. Aspect level sentiment classification with deep memory network[J].arXiv preprint arXiv:1605.08900, 2016.
[28] Chen P, Sun Z, Bing L, et al. Recurrent attention network on memory for aspect sentiment analysis[C]//Proceedings of the Conference on Empirical Methods in Natural Language Processing. 2017: 452-461.
[29] Liu Q, Zhang H, Zeng Y, et al. Content attention model foraspect based sentiment analysis[C]//Proceedings of the World Wide Web Conference. 2018: 1023-1032.

基金

国家自然科学基金(62076175,61976146);江苏省双创博士计划
PDF(5123 KB)

1230

Accesses

0

Citation

Detail

段落导航
相关文章

/