基于联合学习的成分句法与AMR语义分析方法

PDF(2771 KB)

中文信息学报 ›› 2022, Vol. 36 ›› Issue (7) : 13-23.

语言分析与计算

基于联合学习的成分句法与AMR语义分析方法

黄子怡,李军辉,贡正仙

作者信息 +

Improving AMR Parsing & Constituency Parsing with Multi-task Learning

HUANG Ziyi, LI Junhui, GONG Zhengxian

Author information +

History +

摘要

抽象语义表示(Abstract Meaning Representation,AMR)解析任务是从给定的文本中抽象出句子的语义特征,成分句法分析(Constituency Parsing)任务则探寻句子中的层次逻辑结构。由于AMR解析和成分句法分析之间存在着很强的互补性,抽象语义需要把握文本的句法结构,而句法分析可以通过理解句子中的语义信息来避免歧义,因此该文提出了一种联合训练方法用于捕获两个任务之间的内部联系从而提升各自任务的性能。此外,为了解决两个任务由于数据量过少造成的数据依赖问题,该文利用外部语料获得大规模自动标注 AMR 图以及自动标注句法树,并基于上述联合学习方法,采用预训练+微调的半监督学习方法进行训练。实验结果表明,该方法可以有效提高模型的性能,其中AMR解析任务在AMR 2.0上提升了8.73个F₁值,句法分析在PTB上获得了6.36个F₁值的提升。

Abstract

Abstract Semantic Representation Parsing aims to derive the semantic features of sentences from the given text. Constituency Parsing explores the hierarchical syntactic structure in sentences. There is strong complementarity between AMR Parsing and Constituency Parsing, the former utilizes the syntactic structure of the text, and the latter can avoid ambiguity with the help of semantic information. Therefore, this paper proposes a joint learning method to take the advantage from both tasks. Besides, in order to solve the limited data available in two tasks, this paper introduces external corpus to obtain large-scale automatic labeled AMR graph and automatic labeled syntax trees. Experiments show that this method can effectively improve the performance of AMR Parsing by 8.73 increase in F₁-value on AMR2.0 and the performance of Constituency Parsing by 6.36 increase in F₁-value on PTB.

导出引用

黄子怡,李军辉,贡正仙. 基于联合学习的成分句法与AMR语义分析方法. 中文信息学报. 2022, 36(7): 13-23

HUANG Ziyi, LI Junhui, GONG Zhengxian. Improving AMR Parsing & Constituency Parsing with Multi-task Learning. Journal of Chinese Information Processing. 2022, 36(7): 13-23

参考文献

[1] Banarescu L, Bonial C, Cai S, et al. Abstract meaning representation for sembanking[C]//Proceedings of the 7th Linguistic Annotation Workshop and Interoperability With Discourse. 2013: 178-186.
[2] Zhou J, Xu F,Uszkoreit H, et al. AMR parsing with an incremental joint model[C]//Proceedings of the EMNLP, 2016: 680-689.
[3] Jones B, Andreas J, Bauer D, et al. Semantics-based machine translation with hyperedge replacement grammars[C]//Proceedings of the COLING, 2012: 1359-1376.
[4] Bonial C, Donatelli L, Lukin S, et al. Augmenting abstract meaning representation for human-robot dialogue[C]//Proceedings of the 1st International Workshop on Designing Meaning Representations, 2019: 199-210.
[5] Liao K, Lebanoff L, Liu F. Abstract meaning representation for multi-document summarization[C]//Proceedings of the COLING, 2018: 1178-1190.
[6] Ge D, Li J, Zhu M, et al. Modeling source syntax and semantics for neural AMR parsing[C]//Proceedings of the IJCAI, 2019: 4975-4981.
[7] Xu D, Li J, Zhu M, et al. Improving AMR parsing with sequence-to-sequence pre-training[C]//Proceedings of the EMNLP, 2020: 2501-2511.
[8] Suzuki J,Takase S, Kamigaito H, et al. An empirical study of building a strong baseline for constituency parsing[C]//Proceedings of the ACL, 2018: 612-618.
[9] Cross J, Huang L. Span-based constituency parsing with a structure-label system and provably optimal dynamic oracles[C]//Proceedings of the EMNLP, 2016: 1-11.
[10] Stern M, Andreas J, Klein D. A minimal span-based neural constituency parser[C]//Proceedings of the ACL, 2017: 818-827.
[11] Vinyals O, Kaiser , Koo T, et al. Grammar as a foreign language[C]//Proceedings of the NeurIPS, 2015: 2773-2781.
[12] Watanabe T,Sumita E. Transition-based neural constituent parsing[C]//Proceedings of the ACL-IJCNLP, 2015: 1169-1179.
[13] Dyer C,Kuncoro A, Ballesteros M, et al. Recurrent neural network grammars[C]//Proceedings of the NAACL-HLT, 2016: 199-209.
[14] Kitaev N, Klein D. Constituency parsing with a self-attentive encoder[C]//Proceedings of the ACL, 2018: 2676-2686.
[15] Zhang Y, Zhou H, Li Z. Fast and accurate neural crf constituency parsing[C]//Proceedings of the IJCAI, 2020: 4046-4053.
[16] Vaswani A, Shazeer N, Parmar N, et al. Attention is all you need[C]//Proceedings of the NeurIPS, 2017: 5998-6008.
[17] Wang C,Xue N, Pradhan S. Boosting transition-based AMR parsing with refined actions and auxiliary analyzers[C]//Proceedings of the ACL-IJCNLP, 2015: 857-862.
[18] Bauer D. Grammar-based semantic parsinginto graph representations[D]. PhD diss., NY: Columbia University, 2017.
[19] Wang C,Xue N, Pradhan S. A transition-based algorithm for AMR parsing[C]//Proceedings of the NAACL-HLT, 2015: 366-375.
[20] Ballesteros M, Al-onaizan Y. AMR parsing using stack-LSTMs[C]//Proceedings of the EMNLP, 2017: 1269-1275.
[21] Astudillo R F, Ballesteros M, NASEEM T, et al. Transition-based parsing with stack-transformers [C]//Proceedings of the EMNLP, 2020: 1001-1007.
[22] Flanigan J, Thomson S, Carbonell J G, et al. A discriminative graph-based parser for the abstract meaning representation[C]//Proceedings of the ACL, 2014: 1426-1436.
[23] Cai D, Lam W. Core semantic first: a top-down approach for AMR parsing[C]//Proceedings of the EMNLP-IJCNLP, 2019: 3790-3800.
[24] Zhang S, Ma X, Duh K, et al. AMR parsing as sequence-to-graph transduction[C]//Proceedings of the ACL, 2019: 80-94.
[25] Barzdins G, Gosko D. Riga at SemEval-2016 task 8: impact of smatch extensions and character-level neural translation on AMR parsing accuracy[C]//Proceedings of the SemEval, 2016: 1143-1147.
[26] Peng X, Wang C,Gildea D, et al. Addressing the data sparsity issue in neural AMR parsing[C]//Proceedings of the 15th Conference of EACL, 2017: 366-375.
[27] Van Noord R,Bos J. Neural semantic parsing by character-based translation: experiments with abstract meaning representations[J]. Computational Linguistics in the Netherlands Journal, 2017, 7: 93-108.
[28] Konstas I, Iyer S, Yatskar M, et al. Neural AMR: Sequence-to-sequence models for parsing and generation[C]//Proceedings of the 55th ACL, 2017: 146-157.
[29] Chen W T. Learning to map dependency parses to abstract meaning representations[C]//Proceedings of the ACL-IJCNLP Student Research Workshop, 2015: 41-46.
[30] Zhou J, Li Z, Zhao H. Parsing all: Syntax and semantics, dependencies and spans[C]//Proceedings of the EMNLP, 2020: 4438-4449.
[31] Henderson J, Merlo P,Musillo G, et al. A latent variable model of synchronous parsing for syntactic and semantic dependencies[C]//Proceedings of the CoNLL, 2008: 178-182.
[32] Zhu J, Li J, Zhu M, et al. Modeling graph structure in transformer for better AMR-to-text generation[C]//Proceedings of the EMNLP-IJCNLP, 2019: 5462-5471.
[33] He R, Lee W S, Ng H T, et al. An interactive multi-task learning network for end-to-end aspect-based sentiment analysis[C]//Proceedings of the ACL, 2019: 504-515.
[34] Liu X, He P, Chen W, et al. Multi-task deep neural networks for natural language understanding[C]//Proceedings of the ACL, 2019: 4487-4496.
[35] Liu P,Qiu X, Huang X J. Adversarial multi-task learning for text classification[C]//Proceedings of the ACL, 2017: 1-10.
[36] Tenney I, Das D, Pavlick E. BERT rediscovers the classical NLP pipeline[C]//Proceedings of the ACL, 2019: 4593-4601.
[37] Xie Q, Dai Z, Hovy E, et al. Unsupervised data augmentation for consistency training[C]//Proceedings of the NeurIPS, 2020.
[38] Cai S, Knight K. Smatch: An evaluation metric for semantic feature structures[C]//Proceedings of ACL, 2013: 748-752
[39] Zhou Q, Zhang Y, Ji D, et al. AMR parsing with latent structural information[C]//Proceedings of the ACL, 2020: 4306-4319.
[40] Cai D, Lam W. AMR parsing via graph-sequence iterative inference[C]//Proceedings of the ACL, 2020: 1290-1301.
[41] Liu J, Zhang Y. Inorder transition-based constituent parsing[J]. Transactions of the Association for Computational Linguistics, 2017, 5: 413-424.
[42] Vilares D, Abdou M, Sgaard A. Better, faster, stronger sequence tagging constituent parsers[C]//Proceedings of the NAACL-HLT, 2019: 3372-3383.
[43] Mrini K, Dernoncourt F, Tran Q H, et al. Rethinking self-attention: towards interpretability in neural parsing[C]//Proceedings of the EMNLP, 2020: 731-742.

基金

国家自然科学基金(61876120,61976148)

PDF(2771 KB)

1276

Accesses

Citation

Detail

段落导航

摘要
Abstract
关键词
Key words
引用本文
参考文献
基金

Received	Published
2021-01-19	2022-09-01
Issue Date
2022-09-01

选择文件类型/文献管理软件名称

选择包含的内容

摘要

Abstract

关键词

Key words

引用本文

{{custom_sec.title}}

{{custom_sec.title}}

参考文献

{{custom_fnGroup.title_cn}}

脚注

基金