Chinese Complex Sentence Relation Identification Based on Attention Mechanism and Graph Convolutional Network
ZHENG Hao1, LI Yuan1, SHEN Wei2, CHEN Jiajie1
1.School of Computer Science, Central China Normal University, Wuhan, Hubei 430079, China; 2.Central for Research on Language and Language Education, Central China Normal University, Wuhan, Hubei 430079, China
Abstract：Relation identification of complex sentences is to distinguish the categories of semantic relations of sub-sentences. Focused on the sentences without explicit connectives, this paper applies the Attention mechanism combined with the GCN to classify the semantic relationship of Chinese complex sentences. The Bert based sentence representation formed by word vector is input into the Bi-LSTM. The sentence position representation is obtained and weighted via attention mechanism. A graph network is then constructed to capture the semantic information between sentences. The experiments on the CCCS and CDTB datasets reveal that the proposed method achieves 76.2% and 74.4% F1 value, respectively, increasing about 2.1% compared with the previous best models.
[1] 贾旭楠,魏庭新,曲维光,等.基于神经网络的复句判定及其关系识别研究[J].计算机工程,2021,47(11): 8. [2] 胡金柱,舒江波,胡泉,等.复句关系词自动识别中规则的表示方法研究[J].计算机工程与应用,2016, 52(01): 127-132. [3] 胡泉,胡金柱,杨进才,等.现代汉语复句关系词自动识别方法及系统[P].湖北: CN103412855A,2013-11-27. [4] Lin Z, Kan M Y, Ng H T. Recognizing implicit discourse relations in the Penn Discourse Treebank[C]//Proceedings of the Conference on Empirical Methods in Natural Language Processing. 2009: 343-351. [5] Zhou Z M, Xu Y, Niu Z Y, et al. Predicting discourse connectives for implicit discourse relation recognition[C]//Proceedings of the Coling: Posters. 2010: 1507-1514. [6] 郭凤羽,贺瑞芳,党建武.基于语境交互感知和模式筛选的隐式篇章关系识别[J].计算机学报,2020,43(05): 901-915. [7] Yangfeng Ji, Jacob Eisenstein. One vector is not enough: Entity-augmented distributed semantics for discourse relations[J]. Transactions of the Association for Computational Linguistics, 2015, 3: 329-344. [8] Qin L, Zhang Z, Zhao H, et al. Adversarial connective-exploiting networks for implicit discourse relation classification[J]. arXiv preprint arXiv: 1704.00217, 2017. [9] Wang J, Lan M. Two end-to-end shallow discourse parsers for english and chinese in conll-2016 shared task[C]//Proceedings of the CoNLL Shared Task. 2016: 33-40. [10] Cianflone A, Kosseim L. Attention for implicit discourse relation recognition[C]//Proceedings of the Eleventh International Conference on Language Resources and Evaluation. 2018: 1946-1951. [11] Rnnqvist S, Schenk N, Chiarcos C. A recurrent neural model with attention for the recognition of chinese implicit discourse relations[J]. arXiv preprint arXiv: 1704.08092, 2017. [12] Liu Y, Li S. Recognizing implicit discourse relations via repeated reading: Neural networks with multi-level attention[C]//Proceedings of the Conference on Empirical Methods in Natural Language Processing. Austin,USA,2016: 1224-1233. [13] Chen J, Zhang Q, Liu P, et al. Implicit discourse relation detection via a deep architecture with gated relevance network[C]//Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics. Berlin, Germany,2016: 1726-1735. [14] 周文翠,袁春风.并列复句的自动识别初探[J].计算机应用研究,2008(03): 764-766. [15] 洪鹿平.汉语复句关系自动判定研究[D].南京: 南京师范大学硕士学位论文,2008. [16] 吴云芳,石静,万富强,等.汉语并列复句的自动识别方法[J].北京大学学报(自然科学版),2013,49(01): 1-6. [17] 杨进才,涂馨丹,沈显君,等.基于依存关系规则的汉语复句关系词自动识别[J].计算机应用研究,2018,35(06): 1756-1760. [18] Hashimoto K, Stenetorp P, Miwa M, et al. Task-oriented learning of word embeddings for semantic relation classification[J]. arXiv preprint arXiv: 1503.00095, 2015. [19] Yoon Kim. Convolutional neural networks for sentence classification[J]. CoRR,2014,abs/1408.5882. [20] Dongxu Zhang, Dong Wang. Relation classification via recurrent neural network[J]. CoRR,2015,abs/1508.01006. [21] Hochreiter S, Schmidhuber J. Long short-term memory[J]. Neural Computation, 1997, 9(8): 1735-1780. [22] Zhang L, Xiang F. Relation classification via BiLSTM-CNN[C]//Proceedings of the International Conference on Data Mining and Big Data. Springer, Cham, 2018: 373-382. [23] Zhou P, Shi W, Tian J, et al. Attention-based bidirectional long short-term memory networks for relation classification[C]//Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics. 2016: 207-212. [24] Xue N, Ng H T, Pradhan S, et al. Conll shared task on multilingual shallow discourse parsing[C]//Proceedings of the CoNLL Shared Task. 2016: 1-19. [25] Kipf T N, Welling M. Semi-supervised classification with graph convolutional networks[J].arXiv preprint arXiv: 1609.02907, 2016. [26] Marcheggiani D, Titov I. Encoding sentences with graph convolutional networks for semantic role labeling[J]. arXiv preprint arXiv: 1703.04826, 2017. [27] Bastings J, Titov I, Aziz W, et al. Graph convolutional encoders for syntax-aware neural machine translation[J]. arXiv preprint arXiv: 1704.04675, 2017. [28] Yao L, Mao C, Luo Y. Graphconvolutional networks for text classification[J]. Proceedings of the AAAI Conference on Artificial Intelligence, 2019, 33: 7370-7377. [29] 邢福义著. 汉语复句研究[M]. 北京: 商务印书馆, 2001. [30] Cui Y, Che W, Liu T, et al. Pre-training with whole word masking for chinese bert[J].CoRR,2019,abs/1906.08101: [31] Vaswani A, Shazeer N, Parmar N, et al. Attention is all you need[C]//Proceedings of the 31st International Conference on Neural Information Processing Systems. 2017: 5998-6008. [32] 孙凯丽,邓沌华,李源,等. 基于句内注意力机制多路CNN的汉语复句关系识别方法[J].中文信息学报,2020,34(06): 9-17, 26. [33] Zhou Y, Xue N. The Chinese Discourse TreeBank: A Chinese corpus annotated with discourse relations[J]. Language Resources and Evaluation, 2015, 49(2): 397-431.
2022, Vol. 36 
