通常情感分类模型都假定数据集中各类别样本数之间处于平衡状态,实际上数据集中不同类别样本间并不平衡。当样本间存在样本类别不平衡问题时,会导致训练结果偏向多数类样本,少数类样本分类精度不高。另外,在训练过程中,新加入样本存在贡献衰减问题,这将导致新样本对情感分类的效果影响降低,从而影响最终分类效果。针对以上问题,该文提出一种基于混合采样与代价损失再平衡相融合的多通道双向GRU情感分类方法(Re-balance Multichannel Sampling BiGRU, RMS_BiGRU)。该模型首先在数据集上对样本进行混合重采样处理,根据不同的采样形式输入到不同的神经网络通道中,并在各通道中使用损失函数再平衡策略对新老训练样本进行贡献平衡。该文提出的模型可以缓解神经网络对多数类样本的依赖问题,同时样本空间中的所有样本对训练的贡献都大致相同。实验结果表明,该方法在整体G-mean评价上优于其他情感分类方法。
Abstract
Most sentiment classification models assume that the samples in each sentiment category in the dataset is balanced, which may not be true in real practices. In this paper, we propose a multi-channel bidirectional GRU sentiment classification method based on the fusion of mixed sampling and cost-sensitive learning, namely, re-balance multi-channel sampling BiGRU(RMS_BiGRU). We performs mixed resampling strategy on the datasets first. Then, according to different sampling forms we put them into different channels. Meanwhile we use a re-balance strategy in each channel to balance the contributions among new and old training samples. Our method can alleviate the dependency on negative class, and all samples in the sample space almost equally contribute to the training. The experimental results show that this method achieves better classification effect in sentiment classification of different categories.
关键词
情感分类 /
多通道采样 /
再平衡贡献 /
双向GRU
{{custom_keyword}} /
Key words
sentiment classification /
multi-channel sampling /
re-balance contribution /
bidirectional GRU
{{custom_keyword}} /
{{custom_sec.title}}
{{custom_sec.title}}
{{custom_sec.content}}
参考文献
[1] 赵妍妍,秦兵,刘挺.文本情感分析[J].软件学报,2010,21(08):1834-1848.
[2] 殷昊,李寿山,贡正仙,等.基于多通道LSTM的不平衡情绪分类方法[J].中文信息学报,2018,32(01):139-145.
[3] 陈志,郭武.不平衡训练数据下的基于深度学习的文本分类[J].小型微型计算机系统,2020,41(01):1-5.
[4] Kim Y. Convolutional neural networks for sentence classification[J]. arXiv preprint arXiv:1408.5882, 2014.
[5] Mikolov T, Kombrink S, Burget L, et al. Extensions of recurrent neural network language model[C]//Proceedings of the IEEE International Conference on Acoustics, Speech and Signal Processing, 2011: 5528-5531.
[6] Zhu X, Sobihani P, Guo H. Long short-term memory over recursive structures[C]//Proceedings of the International Conference on Machine Learning, 2015: 1604-1612.
[7] Wang Y, Huang M, Zhu X, et al. Attention-based LSTM for aspect-level sentiment classification[C]//Proceedings of the Conference On Empirical Methods In Natural Language Processing, 2016: 606-615.
[8] Tang D, Qin B, Feng X, et al. Effective LSTMs for target-dependent sentiment classification[J]. arXiv preprint arXiv:1512.01100, 2015.
[9] Ma Y, Peng H, Cambria E. Targeted aspect-based sentiment analysis via embedding commonsense knowledge into an attentive LSTM[C]//Proceedings of the 32nd AAAI Conference On Artificial Intelligence, 2018.
[10] Wang Y, Sun A, Huang M, et al. Aspect-level sentiment analysis using as-capsules[C]//Proceedings of the World Wide Web Conference,2019: 2033-2044.
[11] Shams M, Khoshavi N, Baraani Dastjerdi A. LISA: language-independent method for aspect-based sentiment analysis[J]. IEEE Access, 2020, 8: 31034-31044.
[12] Chawla N V, Bowyer K W, Hall L O, et al. SMOTE: Synthetic minority over-sampling technique[J]. Journal of Artificial Intelligence Research, 2002, 16: 321-357.
[13] Ramentol E, Gondres I, Lajes S, et al. Fuzzy-rough imbalanced learning for the diagnosis of high voltage circuit breaker maintenance: The SMOTE-FRST-2T algorithm[J]. Engineering Applications of Artificial Intelligence, 2016, 48: 134-139.
[14] Chan P, Stolfo S J. Toward scalable learning with non-uniform distributions: Effects and a multi-classifier approach[C]//Proceedings of the 4th International Conference on Knowledge Discovery and Data Mining.Citeseer, 1999.
[15] Elkan C. The foundations of cost-sensitive learning[C]//Proceedings of the International Joint Conference on Artificial Intelligence.Lawrence Erlbaum Associates Ltd, 2001: 973-978.
[16] Cui Y, Jia M, Lin T, et al. Class-balanced loss based on effective number of samples[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition,2019: 9268-9277.
[17] Xiao Z, Wang L, Du J Y. Improving the performance of sentiment classification on imbalanced datasets with transfer learning[J]. IEEE Access, 2019, 7: 28281-28290.
[18] Cao K, Wei C, Gaidon A, et al. Learning imbalanced datasets with label-distribution-aware margin loss[C]//Proceedings of Advances in Neural Information Processing Systems, 2019.1567-1578.
[19] Ba J L, Kiros J R, Hinton G E. Layer normalization[J]. arXiv preprint arXiv:1607.06450, 2016.
[20] Pennington J, Socher R, Manning C D. Glove: Global vectors for word representation[C]//Proceedings of the Conference on Empirical Methods in Natural Language Processing, 2014: 1532-1543.
[21] Schuster M, Paliwal K K. Bidirectional recurrent neural networks[J]. IEEE Transactions on Signal Processing, 1997, 45(11): 2673-2681.
[22] Chung J, Gulcehre C, Cho K, et al. Empirical evaluation of gated recurrent neural networks on sequence modeling[J]. arXiv preprint arXiv:1412.3555, 2014.
{{custom_fnGroup.title_cn}}
脚注
{{custom_fn.content}}
基金
国家自然科学基金(62066022);国家重点研发计划(2018YFC0830105)
{{custom_fund}}
PDF(2517 KB)
