基于序列到序列的对话生成在实现情感状态转移时大多采用外部情感词嵌入的方式,编码器很难捕获解码器的情感状态,解码器被强制嵌入的外部情感词干扰,造成生成回复情感词堆叠及缺乏情感信息上下文。为解决上述问题,该文提出基于位置感知的情感可控对话生成模型。在编码的过程中,当前输入词向量和位置向量共同参与编码,在不影响当前输入的情况下,上文信息利用分层的编码方式增加额外编码信息。在解码的过程中,利用遮蔽语言的性能,强制模型进行内容理解和学习,编码器和解码器的联合训练能够生成符合语法的情感回复。实验结果表明,位置感知的加入进一步刻画了数据的潜在结构信息,提高了情感可控对话生成的语言质量。
Abstract
Sequence to sequence generation model mostly uses the way of adding external emotional words when the model transfers the emotional state, which is defected in generation responses with emotional word stacks and lack of emotional information context. To address this issue, this paper proposes an emotion controllable conversation generation model based on position awareness. In the encoding process, the current input word vector and position vector jointly participate in encoding. Without affecting the current input, the preceding context is encoded by an additional leyer. In the decoding process, the masked model is used to force the model to understand and learn the content. The joint training of the encoder and the decoder could generate a grammatical emotion response. The experimental results show that the position awareness further characterizes the potential structural information on the data and improves the model quality.
关键词
对话生成 /
序列到序列模型 /
注意力机制 /
位置感知
{{custom_keyword}} /
Key words
conversation generation /
Seq2Seq model /
attention mechanism /
location awareness
{{custom_keyword}} /
{{custom_sec.title}}
{{custom_sec.title}}
{{custom_sec.content}}
参考文献
[1] Wu Y, Wu W, Yang D, et al. Neural response generation with dynamic vocabularies[C]//Proceedings of the 32nd AAAI Conference on Artificial Intelligence, 2018.
[2] Sun X, Li J, Tao J. Emotional conversation generation orientated syntactically constrained bidirectional-asynchronous framework[C]//Proceedings of the IEEE Transactions on Affective Computing, 2019.
[3] Zhang H, Lan Y, Guo J, et al. Reinforcing coherence for sequence to sequence model in dialogue generation[C]//Proceedings of the IJCAI, 2018: 4567-4573.
[4] Li C, Niu X, Luo X, et al. A review-driven neural model for sequential recommendation[C]//Proceedings of the 28th International Joint Conference on Artificial Intelligence. AAAI Press, 2019: 2866-2872.
[5] Chen H, Yi X, Sun M, et al. Sentiment-controllable chinese poetry generation[C]//Proceedings of the 28th International Joint Conference on Artificial Intelligence. AAAI Press, 2019: 4925-4931.
[6] Zhou, Long, et al. Sequence generation: from both sides to the middle[J]. arXiv preprint arXiv: 1906.09601, 2019.
[7] Mou L, Song Y, Yan R, et al. Sequence to backward and forward sequences: a content-introducing approach to generative short-text conversation[C]//Proceedings the 26th International Conference on Computational Linguistics: Technical Papers, 2016: 3349-3358.
[8] Ma Z, Yang R, Du B, et al. A control unit for emotional conversation generation[J]. IEEE Access, 2020, 8: 43168-43176.
[9] Zhou H, Huang M, Zhang T, et al. emotional chatting machine: Emotional conversation generation with internal and external memory[C]//Proceedings of the AAAI Conference on Artificial Intelligence, 2018.
[10] X. Sun, J. Li and J. Tao, Emotional conversation generation orientated syntactically constrained bidirectional-asynchronous framework[J]. IEEE Transactions on Affective Computing.2022,13(1): 187-198.
[11] Song K, Tan X, Qin T, et al. MASS: masked sequence to sequence pre-training for language generation[C]//Proceedings of the International Conference on Machine Learning, 2019: 5926-5936.
[12] Bahdanau D, Cho K, Bengio Y. Neural machine translation by jointly learning to align and translate[J].arXiv Preprint arXiv: 1409.0473.2014.
[13] Wu X, Zhang T, Zang L, et al. Mask and infill: applying masked language model to sentiment transfer[C]//Proceedings of the 28th International Joint Conference on Artificial Intelligence. AAAI Press, 2019: 5271-5277.
[14] O Vinyals, Q Le. A neural conversational mode, 2015,arXiv: 1506.05869.[OL].Available: https://arxiv.org/abs/1506.05869.
[15] Mikolov T, M Karafiát, Burget L, et al. Recurrent neural network based language model[C]//Proceedings of the International Speech Communication Association, Makuhari, Chiba, Japan, September. 2015: 1045-1048.
[16] S Hochreiter, J Schmidhuber, Long short-term memory[J]. Neural comput, 1997(9), 8: 1735-1780.
[17] K Cho, B van Merrienboer, C Gulcehre D, et al. Learning phrase representations using RNN encoder-decoder for statistical machine translation[J/OL] arXiv preprine, arXiv: 1406.1078, 2014.
[18] Vaswani A, Shazeer N, Parmar N, et al. Attention is all you need[C]//Proceedings of the 31st International Conference on Neural Information Processing Systems, 2017: 5998-6008.
[19] Xu L, Lin H, Pan Y, et al. Constructing the affective lexicon ontology[J]. Journal of the China Society for Scientific and Technical Information, 2008,27(2): 180-185.
[20] Papineni K, Roukos S, Ward T, et al. BLEU: a method for automatic evaluation of machine translation[C]//Proceedings of the 40th Annual Meeting on Association for Computational Lin-guistics. Association for Computational Linguistics, 2002: 311-318.
[21] Liu C W, Lowe R, Serban I V, et al. How NOT to evaluate your dialogue system: an empirical study of unsupervised evaluation metrics for dialogue response generation[C]//Proceedings of the Conference on Empirical Methods in Natural Language Processing, 2016: 2122-2132.
{{custom_fnGroup.title_cn}}
脚注
{{custom_fn.content}}
基金
国家自然科学基金(61762070,61862048);内蒙古自然科学基金(2019MS06004)
{{custom_fund}}
PDF(1640 KB)
