隐式篇章关系识别是一种自动判别论元语义关系的自然语言处理任务。该任务蕴含的关键科学问题涉及两个方面: 其一是准确表征论元语义;其二是基于语义表示,有效地判别论元之间的关系类型。该文将集中在第一个方面开展研究。精准可靠的语义编码有助于关系分类,其根本原因是,编码表示的可靠性促进了正负例样本的可区分性(正例样本特指一对蕴含了“目标关系类”的论元,负例则是一对持有“非目标关系类”的论元)。近期研究显示,集成对比学习机制的语义编码方法能够提升模型在正负例样本上的可辨识性。为此,该文将对比学习机制引入论元语义的表示学习过程,利用“对比损失”驱动正负例样本的“相异性”,即在语义空间中聚合同类正样本,且驱散异类负样本的能力。特别地,该文提出基于条件自编码器的高困惑度负例生成方法,并利用这类负例增强对比学习数据的迷惑性,提升论元语义编码器的鲁棒性。该文使用篇章关系分析的公开语料集PDTB进行实验,实验结果证明,上述方法相较于未采用对比学习的基线模型,在面向对比(Comparison)、偶然(Contingency)、扩展(Expansion)及时序(Temporal)四种PDTB关系类型的二元分类场景中,分别产生了4.68%、4.63%、3.14%、12.77%的F1值性能提升。
Abstract
Implicit discourse relation recognition automatically identifies the semantic relation between arguments. The key to this task involves two issues: one is to represent the argument semantics, the other is to recognize the relation between arguments. Focusing on better representation of the arguments, this paper introduces the contrast learning into the process of argument representation learning. We further propose a method generating confused samples based on conditional auto-encoders, so as to enhance the confused data in contrastive learning. Experiments on the Penn Discourse Treebank (PDTB) corpus show that,our method increases F1 score by 4.68%, 4.63%, 3.14% and 12.77% on four top relations (Comparison, Contingency, Expansion, and Temporal), respectively.
关键词
隐式篇章关系识别 /
对比学习 /
条件变分编码
{{custom_keyword}} /
Key words
implicit discourse relation recognition /
contrastive learning /
condition variational auto-encoder
{{custom_keyword}} /
{{custom_sec.title}}
{{custom_sec.title}}
{{custom_sec.content}}
参考文献
[1] MEYER T, POPESCU BELIS A. Using sense-labeled discourse connectives for statistical machine translation[C]//Proceedings of the ESIRMT/HyTra@EACL. Association for Computational Linguistics, 2012: 129-138.
[2] SOMASUNDARAN S, NAMATA G, WIEBE J, et al. Supervised and unsupervised methods in employing discourse relations for improving opinion polarity classification[C]//Proceedings of the Conference on Empirical Methods in Natural Language Processing, 2009: 170-179.
[3] YOSHIDA Y, SUZUKI J, HIRAO T, et al. Dependency-based discourse parser for single-document summarization[C]//Proceedings of the Conference on Empirical Methods in Natural Language Processing, 2014: 1834-1839.
[4] NARASIMHAN K, BARZILAY R. Machine comprehension with discourse relations[C]//Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing,2015: 1253-1262.
[5] PRASAD R, DINESH N, LEE A, et al. The Penn DiscourseTreeBank 2.0[C]//Proceedings of the International Conference on Language Resource and Evaluation, 2008: 2961-2968.
[6] XU Y, HONG Y, RUAN H, et al. Using active learning to expand training data for implicit discourse relation recognition[C]//Proceedings of the Conference on Empirical Methods in Natural Language Processing, 2018: 725-731.
[7] 朱珊珊, 洪宇, 丁思远, 等. 基于训练样本集扩展的隐式篇章关系分类[J].中文信息学报, 2016, 30(5): 111-120.
[8] GAO T, YAO X, CHEN D. SIMCSE: Simple contrastive learning of sentence embeddings[C]//Proceedings of the Conference on Empirical Methods in NaturalLanguage Processing,2021:6894-6910.
[9] SOHN K, LEE H, YAN X. Learning structured output representation using deep conditional generative models[J]. Advances in Neural Information Processing Systems, 2015, 28: 3483-3491.
[10] SCHROFF F, KALENICHENKO D, PHILBIN J. Facenet: A unified embedding for face recognition and clustering[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2015: 815-823.
[11] VARIA S, HIDEY C, CHAKRABARTY T. Discourse relation prediction: Revisiting word pairs with convolutional networks[C]//Proceedings of the 20th Annual SIGdial Meeting on Discourse and Dialogue, 2019: 442-452.
[12] WU C, SHI X, CHEN Y, et al. Bilingually-constrained synthetic data for implicit discourse relation recognition[C]//Proceedings of the Conference on Empirical Methods in Natural Language Processing, 2016: 2306-2312.
[13] LAN M, WANG J, WU Y, et al. Multi-task attention-based neuralnetworks for implicit discourse relationship representation and identification[C]//Proceedings of the Conference on Empirical Methods in Natural Language Processing, 2017: 1299-1308.
[14] DOU Z, HONG Y, SUN Y, et al. CVAE-based re-anchoring for implicit discourse relation classification[C]//Proceedings of the Findings of the Association for Computational Linguistics: 2021: 1275-1283.
[15] PITLER E, LOUIS A, NENKOVA A. Automatic sense prediction for implicit discourse relations in text[C]//Proceedings of the 47th Annual Meeting of the ACL and the 4th International Joint Conferenceon Natural Language Processing of the AFNLP, 2009:683-691.
[16] LIN Z, KAN M Y, NG H T. Recognizing implicit discourse relations in the Penn Discourse Treebank[C]//Proceedings of the Conference on Empirical Methods in Natural Language Processing, 2009: 343-351.
[17] ZHANG B, SU J, XIONG D, et al. Shallow convolutional neural network for implicit discourse relation recognition[C]//Proceedings of the Conference on Empirical Methods in Natural Language Processing, 2015: 2230-2235.
[18] LIU Y, LI S, ZHANG X, et al. Implicit discourse relation classification via multi-task neural networks[C]//Proceedings of the AAAI Conference on Artificial Intelligence, 2016: 2750-2756.
[19] QIN L, ZHANG Z, ZHAO H, et al. Adversarial connective-exploiting networks for implicit discourse relation classification[C]//Proceedings of the ACL, 2017: 1006-1017.
[20] BAI H, ZHAO H. Deep enhanced representation for implicit discourse relation recognition[G]//Proceeding of the COLING.Association for Computational Linguistics, 2018: 571-583.
[21] DAI Z, HUANG R. Improving implicit discourse relation classification by modeling inter-dependencies of discourse units in aparagraph[C]//Proceedings of the NAACL-HLT, 2018:141-151.
[22] VAN NGO L, THAN K, NGUYEN T H. Employing the correspondence of relations and connectives to identify implicit discours erelations vialable embeddings[C]//Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, 2019: 4201-4207.
[23] ZHANG Y, JIAN P, MENG F, et al. Semantic graph convolutional network for implicit discourse relation classification[J]. arXiv preprint arXiv:1910.09183, 2019.
[24] RUAN H, HONG Y, XU Y, et al. Interactively-propagative attention learning for implicit discourse relation recognition[C]//Proceedings of the 28th International Conference on Computational Linguistics, 2020: 3168-3178.
[25] LI X,HONG Y, RUAN H, et al. Using a penalty-based loss re-estimation method to improve implicit discourse relation classification[C]//Proceedings of the 28th International Conference on Computational Linguistics, 2020: 1513-1518.
[26] LIU X, OU J, SONG Y, et al. On the importance of word and sentence representation learning in implicit discourse relation classification[C]//Proceedings of the IJCAI. Ijcai.org, 2020: 3830-3836.
[27] GUTMANN M, HYVRINEN A. Noise-contrastive estimation: a new estimation principle for unnormalized statistical models[C]//Proceedings of the 13th International Conference on Artificial Intelligence and Statistics. JMLR Workshop and Conference Proceedings, 2010: 297-304.
[28] DROR R, BAUMER G, SHLOMOV S, et al. The hitchiker's guide to testing statistical significance in natural language processing[C]//Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics, 2018: 1383-1392.
[29] Chen J, Zhang Q, Liu P, et al. Implicit discourse relationdetection via a deep architecture with gated relevance network[C]//Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics, 2016: 1726-1735.
{{custom_fnGroup.title_cn}}
脚注
{{custom_fn.content}}
基金
科技部重大专项课题(2020YFB1313601);国家自然科学基金(61773276,62076174)
{{custom_fund}}
PDF(3974 KB)
