Abstract：Essay generation is a challenging task in natural language generation. Unlike poetry and story generation, the essay generation demands accurate sentence-level semantics, clear argumentation structure and reasonable expression of core arguments. The state-of-art solution in this task,retrieval based method ignores the identification of logical argumentation relations, resulting in semantic incoherence and inversion of argumentation logic. In this paper, we apply proposes an argumentation identification method based on explicit semantic structure information, achieving better results than previous natural language inference models on the argumentation identification dataset. At the same time, the argumentation identification result is used as an explicit feature to apply to the sentence ordering model for essay generation, which effectively alleviates the logical inconsistency of the ordering model in the essay generation dataset and further improves the overall performance of the essay generation system.
[1] Radford A, Narasimhan K. Salimans T, et al. Improving language understanding by generativepre-training[R].Technical Report, 2018. [2] 冷海涛. 抽取式作文生成研究[D]. 哈尔滨: 哈尔滨工业大学硕士学位论文, 2018. [3] 吴佳铭,冯骁骋,秦兵. 基于图神经网络的篇章级作文生成[C]. 全国信息检索大会, 2021. [4] Romano L, Kouylekov M, Szpektor I, et al. Investigating a generic paraphrase-based approach for relation extraction[C]//Proceedings of the 11th Conference of the European Chapter of the Association for Computational Linguistics. 2006: 409-416. [5] Conneau A , Kiela D, Schwenk H, et al. Supervised learning of universal sentence representations from natural language inference data[C]//Proceedings of the Conference on Empirical Methods in Natural Language Processing. 2017. [6] Nie Y , Bansal M. Shortcut-stacked sentence encoders for multi-domain inference[C]//Proceedings of the 2nd Workshop on Evaluating Vector Space Representations for NLP, 2017. [7] Chen Q, Zhu X, Ling Z, et al. Enhanced LSTM for natural language inference[J]. arXiv preprint arXiv:1609.06038, 2016. [8] Chen Q, Zhu X, Ling Z, et al. Neural natural language inference models enhanced with external knowledge[C]//Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics, 2018. [9] Kim S, Kang I, Kwak N. Semantic sentence matching with densely-connected recurrent and co-attentive information[C]//Proceedings of the AAAI conference on artificial intelligence. 2019, 33(01): 6586-6593. [10] Peters M, Neumann M, Iyyer M, et al. Deep contextualized word representations[C]//Proceedings of the Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, 2018. [11] Devlin J, Chang M W, Lee K, et al. Bert: Pre-training of deep bidirectional transformers for language understanding[J]. arXiv preprint arXiv:1810.04805, 2018. [12] Yang Z, Dai Z, Yang Y, et al. Xlnet: Generalized autoregressive pretraining for language understanding[C]//Proceedings of the Advances in Neural Information Processing Systems, 2019. [13] Pennington J, Socher R, Manning C. Glove: Global vectors for word representation[C]//Proceedings of the Conference on Empirical Methods in Natural Language Processing, 2014: 1532-1543. [14] Mikolov T, Sutskever I, Chen K, et al. Distributed representations of words and phrases and their compositionality[J]. arXiv preprint arXiv:1310.4546, 2013. [15] Mudrakarta P K, Taly A, Sundararajan M, et al. Did the model understand the question?[J]. arXiv preprint arXiv:1805.05492, 2018. [16] Jia R, Liang P. Adversarial examples for evaluating reading comprehension systems[J]. arXiv preprint arXiv:1707.07328, 2017. [17] Baker C F. The berkley framenet project[C]//Proceedings of COLING, 1998. [18] Palmer M , Gildea D , Kingsbury P. The proposition bank: an annotated corpus of semantic roles[J]. Computational Linguistics, 2005, 31(1):71-106. [19] He L, Lee K, Lewis M, et al. Deep semantic role labeling: what works and whats next[C]//Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics, 2017. [20] Liu T Y. Learning to rank for information retrieval[J]. Acm Sigir Forum, 2010, 41(2):904-904. [21] Chen X, Qiu X, Huang X. Neural sentence ordering[J]. arXiv preprint arXiv:1607.06952, 2016. [22] Gong J, Chen X,Qiu X, et al. End-to-end neural sentence ordering using pointer network[J]. arXiv preprint arXiv:1611.04953, 2016. [23] Zhu Y, Zhou K, Nie J Y, et al. Neural sentence ordering based on constraint graphs[J]. arXiv preprint arXiv:2101.11178, 2021. [24] Yin Y, Lai S, Song L, et al. An external knowledge enhanced graph-based neural network for sentence ordering[J]. Journal of Artificial Intelligence Research, 2021, 70: 545-566. [25] Lin Z, Sun Y, Zhang M. A graph-based neural model for end-to-end frame semantic parsing[J]. arXiv preprint arXiv:2109.12319, 2021. [26] Farahnak F, Kosseim L. Using conditional sentence representation in pointer networks for sentence ordering[C]//Proceedings of the IEEE 15th International Conference on Semantic Computing. IEEE, 2021: 288-295. [27] GitHub:huggingface/transformers[EB/OL]. https://github.com/huggingface/transformers. [28] Hugging Face:bert-base-chinese[EB/OL]. https://huggingface.co/bert-base-chinese. [29] GitHub:HIT-SCIR/ltp[EB/OL]. https://github.com/HIT-SCIR/ltp. [30] Zhang Y, Liu Q, Song L. Sentence-state lstm for text representation[J]. arXiv preprint arXiv:1805.02474, 2018. [31] Zhang Z, Wu Y, Li Z, et al. Explicit contextual semantics for text comprehension[J]. arXiv preprint arXiv:1809.02794, 2018. [32] Lu J, Yang J, Batra D, et al. Hierarchical question-image co-attention for visual question answering[J]. arXiv preprint arXiv:1606.00061, 2016.
2022, Vol. 36 
