指代是一种重要的语言现象,运用指代可以避免复杂的词语在句子中重复出现,使语句简洁连贯。在多轮口语对话中,使用代词指代实体可以提高沟通的效率,然而,对话中频繁出现的代词给计算机语言理解增加了难度,进而影响了机器生成回复的质量。该文提出通过消解代词提高对话生成质量,先通过端到端的共指消解模型识别出多轮对话中蕴含的表述同一实体的所有代词和名词短语,即指代簇(coreference clusters);然后使用两种不同的方法,利用指代簇信息增强对话模型: ①使用指代簇信息恢复问句的完整语义,以降低机器语言理解的难度; ②使用图卷积神经网络将指代簇信息编码融入对话生成模型,以提高机器理解对话的能力。该文所提的两个方法在RiSAWOZ公开数据集上进行了验证,实验结果表明,两个方法均可以显著提升对话生成的性能。
Abstract
Coreference is a common and essential language phenomenon. With coreference, repeated occurrence of complex expressions can be avoided in sentences, which makes sentences concise and coherent. In multi-turn spoken dialogue, the use of pronouns referring to entities can enhance communication efficiency. However, highly-frequent use of pronouns in a dialogue would make it difficult for machine to understand utterances, which in turn affects the quality of machine-generated responses. This article suggests that the quality of dialogue generation can be improved by resolving pronouns, specifically, to identify all the pronouns and noun phrases that express the same entity contained in multi-turn dialogue through coreference resolution model which is defined as coreference clusters. Two different methods are proposed and applied to coreference cluster to improve the dialogue model: (1)Using coreference clusters to recover the complete semantics of a query in order to reduce the difficulty of machine language understanding; (2)Using graph convolutional network to encode the coreference clusters into dialogue model which can improve the language understanding ability of the model. The proposed two methods in this article are tested onRiSAWOZ, a large-scale public dialogue dataset. The experimental results show that both methods can significantly improve the performance of dialogue generation.
关键词
任务型对话系统 /
共指消解 /
图卷积神经网络
{{custom_keyword}} /
Key words
task-oriented dialogue system /
coreference resolution /
graph convolutional network
{{custom_keyword}} /
{{custom_sec.title}}
{{custom_sec.title}}
{{custom_sec.content}}
参考文献
[1] Chen H, Liu X, Yin D, et al. A survey on dialogue systems: Recent advances and new frontiers[J]. Acm sigkdd Explorations Newsletter, 2017, 19(2):25-35.
[2] Quan J,Xiong D, Webber B, et al. GECOR: An end-to-end generative ellipsis and co-reference resolution model for task-oriented dialogue[C]//Proceedings of the Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing. Hong Kong, China: Association for Computational Linguistics, 2019: 4547-4557.
[3] Su H, Shen X, Zhang R, et al. Improving multi-turn dialogue modelling with utterance Rewriter[C]//Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics. Florence, Italy: Association for Computational Linguistics, 2019: 22-31.
[4] Quan J, Zhang S, Cao Q, et al.RiSAWOZ: A large-scale multi-domain wizard-of-Oz dataset with rich semantic annotations for task-oriented dialogue modeling[J]. arXiv preprint arXiv:2010.08738, 2020.
[5] Zheng J, Fang K, Guodong Z. Sequence to sequence model to ellipsis recovery for Chinese short text[J]. Journal of Chinese Information Processing, 2018.
[6] Vinyals O, Fortunato M, Jaitly N. Pointer networks[C]//Proceedings of the International Conference on Neural Information Processing Systems. MIT Press, 2015.
[7] Vaswani A,Shazeer N, Parmar N, et al. Attention is all you need[C]//Proceedings of the 31st Internation Conference on Neural Information Processing Systems, 2017: 5998-6008.
[8] Gu J, Lu Z, Li H, et al. Incorporating copying mechanism in sequence-to-sequence learning[C]//Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics. Berlin, Germany: Association for Computational Linguistics, 2016: 1631-1640.
[9] Lei W,Jin X, Kan M Y, et al. Sequicity: Simplifying task-oriented dialogue systems with single sequence-to-sequence architectures[C]//Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics, 2018: 1437-1447.
[10] Lee K, He L, Lewis M, et al. End-to-end neural coreference resolution[C]//Proceedings of the Conference on Empirical Methods in Natural Language Processing, 2017: 188-197.
[11] Haghighi A, Klein D. Simple coreference resolution with rich syntactic and semantic features[C]//Proceedings of the Conference on Empirical Methods in Natural Language Processing, 2009: 1152-1161.
[12] Lee H,Peirsman Y, Chang A, et al. Stanford's multi-pass sieve coreference resolution system at the CONLL-2011 shared task[C]//Proceedings of the 15th Conference on Computational Natural Language Learning: Shared task. Association for Computational Linguistics, 2011: 28-34.
[13] Durrett G, Klein D. Easyvictories and uphill battles in coreference resolution[C]//Proceedings of the Conference on Empirical Methods in Natural Language Processing, 2013: 1971-1982.
[14] Bjrkelund A, Kuhn J. Learning structured perceptrons for coreference resolution with latent antecedents and non-local features[C]//Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics, 2014: 47-57.
[15] Clark K, Manning C D. Deep reinforcement learning for mention-ranking coreference models[C]//Proceedings of the Conference on Empirical Methods in Natural Language Processing, 2016: 2256-2262.
[16] Clark K, Manning C D. Improving coreference resolution by learning entity-level distributed representations[C]//Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics, 2016: 643-653.
[17] Grav , Bojanowski P, Gupta P, et al. Learning word vectors for 157 languages[C]//Proceedings of the Eleventh International Conference on Language Resources and Evaluation, 2018.
[18] Rudnicky A I, Thayer E, Constantinides P, et al. Creating natural dialogs in the Carnegie Mellon communicator system[C]//Proceedings of the 6th European Conference on Speech Communication and Technology, 1999.
[19] Zue V, Seneff S, Glass J R, et al. JUPlTER: A telephone-based conversational interface for weather information[J]. IEEE Transactions on Speech and Audio Processing, 2000, 8(1): 85-96.
[20] Zue V W, Glass J R. Conversational interfaces: Advances and challenges[J]. Proceedings of the IEEE, 2000, 88(8): 1166-1180.
[21] Eric M, Manning C D. A copy-augmented sequence-to-sequence architecture gives good performance on task-oriented dialogue[C]//Proceedings of the 15th Conference of the European Chapter of the Association for Computational Linguistics, 2017: 468-473.
[22] Eric M, Krishnan L, Charette F, et al. Key-value retrieval networks for task-oriented dialogue[C]//Proceedings of the 18th Annual SIGdial Meeting on Discourse and Dialogue, 2017: 37-49.
[23] Zhang Y,Ou Z, Yu Z. Task-oriented dialog systems that consider multiple appropriate responses under the same context[J]. arXiv preprint arXiv:1911.10484, 2019.
[24] 李鲁.英语预指代词译法浅析[J]. 中国翻译, 1996(1):29-30.
[25] Kipf T N, Welling M. Semi-supervised classification with graph convolutional networks[J]. arXiv preprint arXiv:1609.02907, 2016.
[26] Kingma D, BA J. Adam: A method for stochastic optimization[J]. arXiv: 1412.6980v9, 2017.
{{custom_fnGroup.title_cn}}
脚注
{{custom_fn.content}}
PDF(8059 KB)
