议论文自动生成是自然语言生成中一项极具挑战性的任务,与诗歌、故事等生成任务不同,所生成的文章需要句子语义明确、论证结构清晰并合理地表达出核心论点。上述特点使得现有的预训练模型难以准确地建模并自动生成,因此传统的检索式方法成为解决该问题的主要方式。但前人方法在句子检索和排序过程中只考虑了语义相关度,忽视了对逻辑论证关系的判别,导致语义不连贯、论证逻辑倒置等问题。针对上述问题,该文将自然语言推理应用于论证关系逻辑判别任务,提出了基于显式语义结构的论证关系逻辑判别方法,新模型在论证判别数据集上取得优于以往自然语言推理模型的效果。同时将论文判别结果作为显式特征应用于议论文句子排序模型,在议论文生成数据集中有效改善了排序模型的逻辑不一致问题并进一步提升了议论文生成系统的总体性能。
Abstract
Essay generation is a challenging task in natural language generation. Unlike poetry and story generation, the essay generation demands accurate sentence-level semantics, clear argumentation structure and reasonable expression of core arguments. The state-of-art solution in this task,retrieval based method ignores the identification of logical argumentation relations, resulting in semantic incoherence and inversion of argumentation logic. In this paper, we apply proposes an argumentation identification method based on explicit semantic structure information, achieving better results than previous natural language inference models on the argumentation identification dataset. At the same time, the argumentation identification result is used as an explicit feature to apply to the sentence ordering model for essay generation, which effectively alleviates the logical inconsistency of the ordering model in the essay generation dataset and further improves the overall performance of the essay generation system.
关键词
论证关系 /
句子排序 /
语义结构信息 /
图神经网络
{{custom_keyword}} /
Key words
argumentative relations /
sentence ordering /
semantic structural information /
graph neural network
{{custom_keyword}} /
{{custom_sec.title}}
{{custom_sec.title}}
{{custom_sec.content}}
参考文献
[1] Radford A, Narasimhan K. Salimans T, et al. Improving language understanding by generativepre-training[R].Technical Report, 2018.
[2] 冷海涛. 抽取式作文生成研究[D]. 哈尔滨: 哈尔滨工业大学硕士学位论文, 2018.
[3] 吴佳铭,冯骁骋,秦兵. 基于图神经网络的篇章级作文生成[C]. 全国信息检索大会, 2021.
[4] Romano L, Kouylekov M, Szpektor I, et al. Investigating a generic paraphrase-based approach for relation extraction[C]//Proceedings of the 11th Conference of the European Chapter of the Association for Computational Linguistics. 2006: 409-416.
[5] Conneau A , Kiela D, Schwenk H, et al. Supervised learning of universal sentence representations from natural language inference data[C]//Proceedings of the Conference on Empirical Methods in Natural Language Processing. 2017.
[6] Nie Y , Bansal M. Shortcut-stacked sentence encoders for multi-domain inference[C]//Proceedings of the 2nd Workshop on Evaluating Vector Space Representations for NLP, 2017.
[7] Chen Q, Zhu X, Ling Z, et al. Enhanced LSTM for natural language inference[J]. arXiv preprint arXiv:1609.06038, 2016.
[8] Chen Q, Zhu X, Ling Z, et al. Neural natural language inference models enhanced with external knowledge[C]//Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics, 2018.
[9] Kim S, Kang I, Kwak N. Semantic sentence matching with densely-connected recurrent and co-attentive information[C]//Proceedings of the AAAI conference on artificial intelligence. 2019, 33(01): 6586-6593.
[10] Peters M, Neumann M, Iyyer M, et al. Deep contextualized word representations[C]//Proceedings of the Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, 2018.
[11] Devlin J, Chang M W, Lee K, et al. Bert: Pre-training of deep bidirectional transformers for language understanding[J]. arXiv preprint arXiv:1810.04805, 2018.
[12] Yang Z, Dai Z, Yang Y, et al. Xlnet: Generalized autoregressive pretraining for language understanding[C]//Proceedings of the Advances in Neural Information Processing Systems, 2019.
[13] Pennington J, Socher R, Manning C. Glove: Global vectors for word representation[C]//Proceedings of the Conference on Empirical Methods in Natural Language Processing, 2014: 1532-1543.
[14] Mikolov T, Sutskever I, Chen K, et al. Distributed representations of words and phrases and their compositionality[J]. arXiv preprint arXiv:1310.4546, 2013.
[15] Mudrakarta P K, Taly A, Sundararajan M, et al. Did the model understand the question?[J]. arXiv preprint arXiv:1805.05492, 2018.
[16] Jia R, Liang P. Adversarial examples for evaluating reading comprehension systems[J]. arXiv preprint arXiv:1707.07328, 2017.
[17] Baker C F. The berkley framenet project[C]//Proceedings of COLING, 1998.
[18] Palmer M , Gildea D , Kingsbury P. The proposition bank: an annotated corpus of semantic roles[J]. Computational Linguistics, 2005, 31(1):71-106.
[19] He L, Lee K, Lewis M, et al. Deep semantic role labeling: what works and whats next[C]//Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics, 2017.
[20] Liu T Y. Learning to rank for information retrieval[J]. Acm Sigir Forum, 2010, 41(2):904-904.
[21] Chen X, Qiu X, Huang X. Neural sentence ordering[J]. arXiv preprint arXiv:1607.06952, 2016.
[22] Gong J, Chen X,Qiu X, et al. End-to-end neural sentence ordering using pointer network[J]. arXiv preprint arXiv:1611.04953, 2016.
[23] Zhu Y, Zhou K, Nie J Y, et al. Neural sentence ordering based on constraint graphs[J]. arXiv preprint arXiv:2101.11178, 2021.
[24] Yin Y, Lai S, Song L, et al. An external knowledge enhanced graph-based neural network for sentence ordering[J]. Journal of Artificial Intelligence Research, 2021, 70: 545-566.
[25] Lin Z, Sun Y, Zhang M. A graph-based neural model for end-to-end frame semantic parsing[J]. arXiv preprint arXiv:2109.12319, 2021.
[26] Farahnak F, Kosseim L. Using conditional sentence representation in pointer networks for sentence ordering[C]//Proceedings of the IEEE 15th International Conference on Semantic Computing. IEEE, 2021: 288-295.
[27] GitHub:huggingface/transformers[EB/OL]. https://github.com/huggingface/transformers.
[28] Hugging Face:bert-base-chinese[EB/OL]. https://huggingface.co/bert-base-chinese.
[29] GitHub:HIT-SCIR/ltp[EB/OL]. https://github.com/HIT-SCIR/ltp.
[30] Zhang Y, Liu Q, Song L. Sentence-state lstm for text representation[J]. arXiv preprint arXiv:1805.02474, 2018.
[31] Zhang Z, Wu Y, Li Z, et al. Explicit contextual semantics for text comprehension[J]. arXiv preprint arXiv:1809.02794, 2018.
[32] Lu J, Yang J, Batra D, et al. Hierarchical question-image co-attention for visual question answering[J]. arXiv preprint arXiv:1606.00061, 2016.
{{custom_fnGroup.title_cn}}
脚注
{{custom_fn.content}}
基金
国家重点研发计划项目(2018YFB1005103)
{{custom_fund}}
PDF(7069 KB)
