近年来,图神经网络模型因其对非欧氏数据的建模和对全局依赖关系的捕获能力而广泛应用于文本分类任务。现有的基于图卷积网络的分类模型中的构图方法存在消耗内存过大、难以适应新文本等问题。此外,现有研究中用于描述图节点间的全局依赖关系的方法并不完全适用于分类任务。为解决上述问题,该文设计并提出了基于概率分布的文本分类网络模型,以语料库中的词和标签为节点构建标签-词异构关系图,利用词语在各标签上的概率分布描述节点间的全局依赖关系,并通过图卷积操作进行文本表示学习。在5个公开的文本分类数据集上的实验表明,该文提出的模型在有效缩减图尺寸的同时,相比于其他文本分类网络模型取得了较为先进的结果。
Abstract
In recent years, graph neural network model has been widely used in text classification tasks because of its ability to model non Euclidean data and capture global dependencies. In existing classification models based on graph convolution network, the composition method consumes too much memory and is difficult to adapt to new text. In addition, the existing methods of describing the global dependencies between graph nodes are not completely suitable for classification tasks. In order to solve the above problems, a probability distribution based graph convolution network for text classification is proposed. A heterogeneous relationship graph is constructed with words and labels in corpus as nodes. The global dependency relation between nodes is described by the probability distribution of words on each label, and the text representation learning is carried out by graph convolution. Experiments on 5 open text classification datasets show that the proposed model achieves better results wth a reduced graph size compared with other text classification network models.
关键词
自然语言处理 /
文本分类 /
深度学习 /
图卷积网络
{{custom_keyword}} /
Key words
natural language processing /
text classification /
deep learning /
graph convolution network
{{custom_keyword}} /
{{custom_sec.title}}
{{custom_sec.title}}
{{custom_sec.content}}
参考文献
[1] Minaee S, Kalchbrenner N, Cambria E, et al. Deep learning-based text classification: a comprehensive review[J]. ACM Computing Surveys (CSUR), 2021, 54(3): 1-40.
[2] Carreras X, Marquez L S, Salgado J G. Boosting trees for anti-spam email filtering[C]//Proceedings of RANLP-01, 4th International Conference on Recent Advances in Natural Language Processing, Tzigov Chark, BG, 2001.
[3] Melville P, Gryc W, Lawrence R D. Sentiment analysis of blogs by combining lexical knowledge with text classification[C]//Proceedings of the 15th ACM SIGKDD international conference on Knowledge Discovery and Data Mining, 2009: 1275-1284.
[4] Hashemi H B, Asiaee A, Kraft R. Query intent detection using convolutional neural networks[C]//Proceedings of the International Conference on Web Search and Data Mining, Workshop on Query Understanding, 2016.
[5] Li Q, Peng H, Li J, et al. A survey on text classification: from shallow to deep learning[J]. arXiv preprint arXiv:2008.00364, 2020.
[6] Kim Y. Convolutional neural networks for sentence classification[C]//Proceedings of the Conference on Empirical Methods in Natural Language Processing, Stroudsburg, PA: ACL, 2014:1746-1751.
[7] Liu P, Qiu X, Huang X, et al. Recurrent neural network for text classification with multi-task learning[C]//Proceedings of the International Joint Conference on Artificial Intelligence, New York, USA: AAAI, 2016:2873-2879.
[8] Bahdanau D, Cho K H, Bengio Y. Neural machine translation by jointly learning to align and translate[C]//Proceedings of the 3rd International Conference on Learning Representations, 2015.
[9] Vaswani A, Shazeer N, Parmar N, et al. Attention is all you need [C]//Proceedings of the 31st International Conference on Neural Information Processing Systems, 2017: 5998-6008.
[10] Defferrard M, Bresson X, Vandergheynst P. Convolutional neural networks on graphs with fast localized spectral filtering[C]//Proceedings of the 30th Conference on Neural Information Processing Systems, 2016, 29.
[11] Bastings J, Titov I, Aziz W, et al. Graph convolutional encoders for syntax-aware neural machine translation[C]//Proceedings of the Conference on Empirical Methods in Natural Language Processing, 2017: 1957-1967.
[12] Marcheggiani D, Titov I. Encoding sentences with graph convolutional networks for semantic role labeling[C]//Proceedings of the Conference on Empirical Methods in Natural Language Processing, 2017: 1506-1515.
[13] 肖驰,徐林莉.松耦合图卷积文本分类网络[J].小型微型计算机系统,2021,42(03): 449-453.
[14] Zhou J, Cui G, Hu S, et al. Graph neural networks: A review of methods and applications[J]. AI Open, 2020, 1: 57-81.
[15] Ying R, He R, Chen K, et al. Graph convolutional neural networks for web-scale recommender systems[C]//Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, 2018: 974-983.
[16] Gokccer Y, Demirci M F, Tan M. A graph-based pattern recognition for chemical molecule matching[C]//Proceedings of the International Conference on Bioinformatics Models. Setubal, Portugal: SciTePress, 2015: 158-162.
[17] Wang Q, Mao Z, Wang B, et al. Knowledge graph embedding: a survey of approaches and applications[J]. IEEE Transactions on Knowledge and Data Engineering, 2017, 29(12): 2724-2743.
[18] Hamilton W L, Ying R, Leskovec J. Inductive representation learning on large graphs[C]//Proceedings of the 31st International Conference on Neural Information Processing Systems, 2017: 1025-1035.
[19] Kipf T N, Welling M. Semi-supervised classification with graph convolutional networks [C]//Proceedings of the 5th International Conference on Learning Representations, Toulon, France, 2017.
[20] Veliˇckovi＇cP, Cucurull G, Casanova A, et al. Graph attention networks [C]//Proceedings of the 6th International Conference on Learning Representations, Vancouver, Canada, 2018: 1-12.
[21] Yao L, Mao C, Luo Y. Graph convolutional networks for text classification[C]//Proceedings of the 33rd AAAI Conference on Artificial Intelligence, Honolulu, USA: AAAI, 2019: 7370-7377.
[22] Zhang Y, Yu X, Cui Z, et al. Every document owns Its structure: inductive text classification via graph neural networks[C]//Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, 2020: 334-339.
[23] Liu X, You X, Zhang X, et al. Tensor graph convolutional networks for text classification[C]//Proceedings of the AAAI Conference on Artificial Intelligence, 2020, 34(05): 8409-8416.
[24] He K, Zhang X, Ren S, et al. Deep residual learning for image recognition[C]//Proceedings of the IEEE conference on computer vision and pattern recognition, 2016: 770-778.
[25] Joulin A, Grave , Bojanowski P, et al. Bag of tricks for efficient text classification[C]//Proceedings of the 15th Conference of the European Chapter of the Association for Computational Linguistics, 2017: 427-431.
[26] Li W, Li S, Ma S, et al. Recursive graphical neural networks for text classification[J]. arXiv preprint arXiv:1909.08166, 2019.
{{custom_fnGroup.title_cn}}
脚注
{{custom_fn.content}}
基金
内蒙古自治区高等学校科学研究项目(NJZZ21004),内蒙古自治区自然科学基金(2021LHMS06009)
{{custom_fund}}
PDF(6070 KB)
