大多数文本分析方法未能提取足够的上下文文本信息与关键特征信息,该文提出BC-CapsNet模型来提取更多特征以进一步提高文本分类准确度。首先使用BERT预训练模型对文本进行词嵌入,然后使用双通道模型与胶囊网络(Capsule Network)进行特征提取,一个通道使用双向门限循环单元(BiGRU)提取上下文文本信息,另一个通道使用卷积神经网络(CNN)捕捉文本的关键特征;最后将两通道提取的特征进行融合并送入到胶囊网络中,胶囊网络使用矢量信息进行特征表示,其与传统网络的标量特征信息相比更具表现力。同时在胶囊网络中,动态路由算法可以提取更多隐藏的特征信息,从而提高文本分类效果。在THUCNews与Ag_News文本数据集上进行的大量实验表明,该模型能够有效地提高文本分类的准确率。
Abstract
This paper proposes BC-CapsNet model to extract more features to further improve the accuracy of text classification.The BERT model is used to embed words in the text, and the dual channel model and capsule network are used for feature extraction. One channel uses bidirectional threshold cyclic unit (BiGRU) to extract the context text information, and the other channel uses convolutional neural network (CNN) to capture the key features of the text. The features extracted by the two channels are finally fused and sent to the capsule network. Experiments on datasets of THUCNews and Ag_News show that the model can effectively improve the accuracy of text classification.
关键词
文本分类 /
胶囊网络 /
神经网络
{{custom_keyword}} /
Key words
text classification /
capsule network /
neural network
{{custom_keyword}} /
{{custom_sec.title}}
{{custom_sec.title}}
{{custom_sec.content}}
参考文献
[1] ZHANG Q, GAO T, LIU X, et al. Public environment emotion prediction model lstm network[OL]. Sustainability, 2020,12 (4): 1665.
[2] ZULQARNAIN M, GHAZALI R, HASSIM Y, et al. Text classification based on gated recurrent unit combines with support vector machine[J].International Journal of Electrical and Computer Engineering, 2020,10(4): 3734.
[3] 刘慧清,郭延哺,李维华.基于贝叶斯网的跨领域情感分析方法[J].计算机应用与软件,2020,37(12): 119-126.
[4] WAWRE S V, DESHMUKH S N. Sentiment classifition using machine learning techniques[J].International Journal of Science and Research,2016, 5(4): 819-821.
[5] SABOUR S, FROSST N, HINTON G E. Dynamic routing between capsules[C]//Proceedings of Advances in Neural Information Processing Systems,2017,26: 3859-3869.
[6] MIKOLOV T, SUTSKEVER I, KAI C, et al. Distrib uted representations of words and phrases and their compositionality[C]//Proceedings of Advances in Neural Information Processing Systems, 2013: 3111-3119.
[7] PENNINGTON J, SOCHER R, MANNING C. Glove: Global vectors for word representation[C]//Proceedings of the Conference on Empirical Methods in Natural Language Processing, 2014: 1532-1543.
[8] VASWANI A, SHAZEER N, PARMAR N, et al. Attetion is all you need[C]//Proceedings of 31st Conference on Neural Information Processing Systems, 2017: 6000-6010.
[9] DEVLIN J, CHANG M W, LEE K, et al. BERT: Pre-training of deep bidirectional transformers for language understanding[C]//Proceedings of the Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, 2019: 4171-4186.
[10] KIM Y. Convolutional neural networks for sen tence classification[C]//Proceedings of the Conference on Empirical Methods in Natural Language Processing,2014: 1746-1751.
[11] 朱烨,陈世平.最近邻注意力和卷积神经网络的文本分类模型[J].小型微型计算机系统,2020,41(02): 375-380.
[12] 王海涛,宋文,王辉.一种基于LSTM和CNN混合模型的文本分类方法[J]. 小型微型计算机系统, 2020,
41(0 6): 1163-1168.
[13] 王伟,孙玉霞,齐庆杰,等. 基BiGRU-attention神经网络的文本情感分类模型[J].计算机应用研究, 2019,36(12): 3558-3564.
[14] HINTON G E, SABOUR S, FROSST N. Matrix capsules with EM routing[C]//Proceedings of 6th International Conference on Learning Representations, 2018: 1-15.
[15] ZHAO W, YE J, YANG M, et al. Investigating capsule networks with dynamic routing for text classification[C]//Proceedings of Association for Computational Linguistics, 2018: 3110-3119.
[16] REN H, LU H. Compositional coding capsule network with k-means routing for text classification[OL].https://arxiv.org/abs/1810.09177.
[17] SU J, YU S, LUO D. Enhancing aspect-based sentiment analysis with capsule network[J]. IEEE Access, 2020, 8: 100551-100561.
[18] WANG Y, SUN A, HUANG M, et al. Aspect-level sentiment analysis using as-capsules[C]//Proceedings of the World Wide Web Conference, 2019: 2033-2044.
[19] DAI J, CHEN C. Text classification system of academic papers based on hybrid Bert-BiGRU model[C]//Proceedings of International Conference on Intelligent Human Machine Systems and Cybernetics, 2020: 40-44.
[20] NGUYEN T H, GRISHMAN R. Relation extraction: Perspective from convolutional neural networks[C]//Proceedings of Workshop on Vector Space Modeling for Natural Language Processing, 2015: 39-48.
{{custom_fnGroup.title_cn}}
脚注
{{custom_fn.content}}
基金
国家自然科学基金(61703277)
{{custom_fund}}
PDF(2492 KB)
