复句的关系识别是为了区分句子语义关系的类别,是自然语言处理(NLP)中必不可少的基础研究任务。现有研究无法使机器在表层判别缺少显式句间连接词句子的语义关系类型。该文将Attention机制与图卷积神经网络(GCN)相结合应用到汉语复句语义关系识别中,通过BERT预训练模型获取单句词向量,输入到Bi-LSTM获取句子位置表示,经Attention机制得到各位置间权重构建图网络以捕获句子间的语义信息,通过图卷积抽取深层的关联信息。该文的方法对缺少显式句间连接词句子的关系识别达到了较好的识别效果,为进一步研究计算机自动分析、识别处理复句的基本方法奠定基础。实验结果表明,在汉语复句语料库(CCCS)和汉语篇章树库(CDTB)数据集上,与先前最好的模型相比,其准确率分别为77.3%和75.7%,提升约1.6%,宏平均F1值分别为76.2%和74.4%,提升约2.1%,说明了该文方法的有效性。
Abstract
Relation identification of complex sentences is to distinguish the categories of semantic relations of sub-sentences. Focused on the sentences without explicit connectives, this paper applies the Attention mechanism combined with the GCN to classify the semantic relationship of Chinese complex sentences. The Bert based sentence representation formed by word vector is input into the Bi-LSTM. The sentence position representation is obtained and weighted via attention mechanism. A graph network is then constructed to capture the semantic information between sentences. The experiments on the CCCS and CDTB datasets reveal that the proposed method achieves 76.2% and 74.4% F1 value, respectively, increasing about 2.1% compared with the previous best models.
关键词
关系识别 /
图卷积神经网络 /
注意力机制 /
Bi-LSTM
{{custom_keyword}} /
Key words
relation classification /
graph convolutional neural network /
attention mechanism /
Bi-LSTM
{{custom_keyword}} /
{{custom_sec.title}}
{{custom_sec.title}}
{{custom_sec.content}}
参考文献
[1] 贾旭楠,魏庭新,曲维光,等.基于神经网络的复句判定及其关系识别研究[J].计算机工程,2021,47(11): 8.
[2] 胡金柱,舒江波,胡泉,等.复句关系词自动识别中规则的表示方法研究[J].计算机工程与应用,2016, 52(01): 127-132.
[3] 胡泉,胡金柱,杨进才,等.现代汉语复句关系词自动识别方法及系统[P].湖北: CN103412855A,2013-11-27.
[4] Lin Z, Kan M Y, Ng H T. Recognizing implicit discourse relations in the Penn Discourse Treebank[C]//Proceedings of the Conference on Empirical Methods in Natural Language Processing. 2009: 343-351.
[5] Zhou Z M, Xu Y, Niu Z Y, et al. Predicting discourse connectives for implicit discourse relation recognition[C]//Proceedings of the Coling: Posters. 2010: 1507-1514.
[6] 郭凤羽,贺瑞芳,党建武.基于语境交互感知和模式筛选的隐式篇章关系识别[J].计算机学报,2020,43(05): 901-915.
[7] Yangfeng Ji, Jacob Eisenstein. One vector is not enough: Entity-augmented distributed semantics for discourse relations[J]. Transactions of the Association for Computational Linguistics, 2015, 3: 329-344.
[8] Qin L, Zhang Z, Zhao H, et al. Adversarial connective-exploiting networks for implicit discourse relation classification[J]. arXiv preprint arXiv: 1704.00217, 2017.
[9] Wang J, Lan M. Two end-to-end shallow discourse parsers for english and chinese in conll-2016 shared task[C]//Proceedings of the CoNLL Shared Task. 2016: 33-40.
[10] Cianflone A, Kosseim L. Attention for implicit discourse relation recognition[C]//Proceedings of the Eleventh International Conference on Language Resources and Evaluation. 2018: 1946-1951.
[11] Rnnqvist S, Schenk N, Chiarcos C. A recurrent neural model with attention for the recognition of chinese implicit discourse relations[J]. arXiv preprint arXiv: 1704.08092, 2017.
[12] Liu Y, Li S. Recognizing implicit discourse relations via repeated reading: Neural networks with multi-level attention[C]//Proceedings of the Conference on Empirical Methods in Natural Language Processing. Austin,USA,2016: 1224-1233.
[13] Chen J, Zhang Q, Liu P, et al. Implicit discourse relation detection via a deep architecture with gated relevance network[C]//Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics. Berlin, Germany,2016: 1726-1735.
[14] 周文翠,袁春风.并列复句的自动识别初探[J].计算机应用研究,2008(03): 764-766.
[15] 洪鹿平.汉语复句关系自动判定研究[D].南京: 南京师范大学硕士学位论文,2008.
[16] 吴云芳,石静,万富强,等.汉语并列复句的自动识别方法[J].北京大学学报(自然科学版),2013,49(01): 1-6.
[17] 杨进才,涂馨丹,沈显君,等.基于依存关系规则的汉语复句关系词自动识别[J].计算机应用研究,2018,35(06): 1756-1760.
[18] Hashimoto K, Stenetorp P, Miwa M, et al. Task-oriented learning of word embeddings for semantic relation classification[J]. arXiv preprint arXiv: 1503.00095, 2015.
[19] Yoon Kim. Convolutional neural networks for sentence classification[J]. CoRR,2014,abs/1408.5882.
[20] Dongxu Zhang, Dong Wang. Relation classification via recurrent neural network[J]. CoRR,2015,abs/1508.01006.
[21] Hochreiter S, Schmidhuber J. Long short-term memory[J]. Neural Computation, 1997, 9(8): 1735-1780.
[22] Zhang L, Xiang F. Relation classification via BiLSTM-CNN[C]//Proceedings of the International Conference on Data Mining and Big Data. Springer, Cham, 2018: 373-382.
[23] Zhou P, Shi W, Tian J, et al. Attention-based bidirectional long short-term memory networks for relation classification[C]//Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics. 2016: 207-212.
[24] Xue N, Ng H T, Pradhan S, et al. Conll shared task on multilingual shallow discourse parsing[C]//Proceedings of the CoNLL Shared Task. 2016: 1-19.
[25] Kipf T N, Welling M. Semi-supervised classification with graph convolutional networks[J].arXiv preprint arXiv: 1609.02907, 2016.
[26] Marcheggiani D, Titov I. Encoding sentences with graph convolutional networks for semantic role labeling[J]. arXiv preprint arXiv: 1703.04826, 2017.
[27] Bastings J, Titov I, Aziz W, et al. Graph convolutional encoders for syntax-aware neural machine translation[J]. arXiv preprint arXiv: 1704.04675, 2017.
[28] Yao L, Mao C, Luo Y. Graphconvolutional networks for text classification[J]. Proceedings of the AAAI Conference on Artificial Intelligence, 2019, 33: 7370-7377.
[29] 邢福义著. 汉语复句研究[M]. 北京: 商务印书馆, 2001.
[30] Cui Y, Che W, Liu T, et al. Pre-training with whole word masking for chinese bert[J].CoRR,2019,abs/1906.08101:
[31] Vaswani A, Shazeer N, Parmar N, et al. Attention is all you need[C]//Proceedings of the 31st International Conference on Neural Information Processing Systems. 2017: 5998-6008.
[32] 孙凯丽,邓沌华,李源,等. 基于句内注意力机制多路CNN的汉语复句关系识别方法[J].中文信息学报,2020,34(06): 9-17, 26.
[33] Zhou Y, Xue N. The Chinese Discourse TreeBank: A Chinese corpus annotated with discourse relations[J]. Language Resources and Evaluation, 2015, 49(2): 397-431.
{{custom_fnGroup.title_cn}}
脚注
{{custom_fn.content}}
基金
国家社会科学基金(18BYY174,19BYY092)
{{custom_fund}}
PDF(3432 KB)
