意图识别和槽填充的联合模型将口语语言理解(Spoken Language Understanding, SLU)提升到了一个新的水平,但是现有模型通过话语上下文信息判断位置信息,缺少对槽信息标签之间位置信息的考虑,导致模型在槽位提取过程中容易发生边界错误,进而影响最终槽位提取表现。此外,在槽信息提取任务中,槽指称项(Slot mentions)可能与正常表述话语并没有区别,特别是电影名字、歌曲名字等,模型容易受到槽指称项话语的干扰,因而无法在槽位提取中正确识别槽位边界。该文提出了一种面向口语语言理解的结合边界预测和动态模板的槽填充(Boundary-prediction and Dynamic-template Slot Filling, BDSF)模型。该模型提供了一种联合预测边界信息的辅助任务,将位置信息引入到槽信息填充中,同时利用动态模板机制对话语句式建模,能够让模型聚焦于话语中的非槽指称项部分,避免了模型被槽指称项干扰,增强模型区分槽位边界的能力。在公共基准语料库SMP-ECDT和CAIS上的实验结果表明,该模型优于对比模型,特别是能够为槽标签预测模型提供准确的位置信息。
Abstract
The joint model of intent detection and slot filling promotes the spoken language understanding (SLU) to a new level. However, the existing models determine position information only by utterances context, ignoring the location information between slot information tags. This paper presents a slot filling model which combines boundary prediction and dynamic template (BDSF). It provides an auxiliary task of joint prediction of boundary information, and introduces location information into slot filling. Moreover, the model can focus on the non-slot mentions utterances via modeling the dialogue sentence pattern by dynamic template mechanism. This method avoids the interference of the slot mentions of the model and enhances the ability of the model to distinguish the slot boundary. The experimental results on the public benchmark corpus SMP-ECDT and CAIS show that the proposed model is better than the comparison models.
关键词
口语语言理解 /
槽填充 /
位置信息 /
模板机制
{{custom_keyword}} /
Key words
spoken language understanding /
slot filling /
position information /
template mechanism
{{custom_keyword}} /
{{custom_sec.title}}
{{custom_sec.title}}
{{custom_sec.content}}
参考文献
[1] TR G, MORI R. Spoken language understanding: Systems for extracting semantic information from speech [M]. John Wiley and Sons, Inc, 2011.
[2] 俞凯,陈露,陈博,等. 任务型人机对话系统中的认知技术: 概念、进展及其未来[J]. 计算机学报, 2015, 38(12): 2333-2348.
[3] 赵阳洋,王振宇,王佩,等. 任务型对话系统研究综述[J]. 计算机学报, 2020, 43(10): 1862-1896.
[4] LIU B, LANE I . Attention-based recurrent neural network models for joint intent detection and slot filling[C]//Proceedings of the 17th Annual Conference of the International Speech Communication Association, 2016: 685-689.
[5] ZHANG X, WANG H. A joint model of intent determination and slot filling for spoken language understanding[C]//Proceedings of the 25th International Joint Conference on Artificial Intelligence, 2016: 2993-2999.
[6] LI C, LI L, QI J. A self-attentive model with gate mechanism for spoken language understanding[C]//Proceedings of the Conference on Empirical Methods in Natural Language Processing, 2018: 3824-3833.
[7] GOO C,GAO G, HSU Y, et al. Slot-gated modeling for joint slot filling and intent prediction[C]//Proceedings of the Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, 2018: 753-757.
[8] E H, NIU P, CHEN Z, et al. A novel bi-directional interrelated model for joint intent detection and slot filling[C]//Proceedings of the 57th Conference of the Association for Computational Linguistics, 2019: 5467-5471.
[9] QIN L, CHE W, LI Y, et al. A stack-propagation framework with token-level intent detection for spoken language understanding[C]//Proceedings of the Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing, 2019: 2078-2087.
[10] QIN L, LIU T, CHE W, et al. A Co-interactive transformer for joint slot filling and intent detection[C]//Proceedings of the 46th IEEE International Conference on Acoustics, Speech and Signal Processing, 2021: 8193-8197.
[11] WANG J, WEI K, RADFAR M, et al. Encoding syntactic knowledge in transformer encoder for intent detection and slot filling[C]//Proceedings of the 35th AAAI Conference on Artificial Intelligence, 2021: 13943-13950.
[12] TENG D, QIN L, CHE W, et al. Injecting word information with multi-level word adapter for Chinese spoken language understanding[C]//Proceedings of the 46th IEEE International Conference on Acoustics, Speech and Signal Processing, 2021: 8188-8192.
[13] LIU Z, WINATA G, XU P, et al. Coach: A coarse-to-fine approach for cross-domain slot filling[C]//Proceedings of the 58th Annual Meeting of the Association for ComputationalLinguistics, 2020: 19-25.
[14] HAFFNER P, TR G, WRIGHT J. Optimizing SVMs for complex call classification[C]//Proceedings of the IEEE International Conference on Acoustics, Speech, and Signal Processing, 2003: 632-635.
[15] SCHAPIRE R, SINGER Y. Boostexter: A boosting-based system for text categorization [J]. Machine Learning, 2000, 39(2/3): 135-168.
[16] RAYMOND C, RICCARDI G. Generative and discriminative algorithms for spoken language understanding[C]//Proceedings of the 8th Annual Conference of the International Speech Communication Association, 2007: 1605-1608.
[17] XU P, SARIKAYA R. Convolutional neural network based triangular CRF for joint intent detection and slot filling[C]//Proceedings of the IEEE Workshop on Automatic Speech Recognitionand Understanding, 2013: 78-83.
[18] MESNIL G, DAUPHIN Y, YAO K, et al. Using recurrent neural networks for slot filling in spoken language understanding [J]. IEEE/ACM Transactions on Audio, Speech, and Language Processing, 2015, 23(3): 530-539.
[19] YAO K, PENG B, ZHANG Y, et al. Spoken language understanding using long short-term memory neural networks[C]//Proceedings of IEEE Spoken Language Technology Workshop, 2014: 189-194.
[20] YAO K, PENG B, ZWEIG G, et al. Recurrent conditional random field for language understanding[C]//Proceedings of the 39th IEEE International Conference on Acoustics, Speech and Signal Processing, 2014: 4077-4081.
[21] DEVLIN J, CHANG M, LEE K, et al. BERT: Pre-training of deep bidirectional transformers for language understanding[C]//Proceedings of the Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, 2019: 4171-4186.
[22] CHEN Q, ZHUO Z, WANG W. BERT for joint intent classification and slot filling [J/OL]. arXiv preprint arXiv: 1902.10909, 2019.
[23] ZHANG C, LI Y,DU N, et al. Joint slot filling and intent detection via capsule neural networks [C]//Proceedings of the 57th Conference of the Association for Computational Linguistics, 2019: 5259-5267.
[24] HOCHREITER S, SCHMIDHUBER J. Long short-term memory [J]. Neural Computation, 1997, 9(8): 1735-1780.
[25] VASWANI A,SHAZEER N, PARMAR N, et al. Attention is all you need [C]//Proceedings of the 31st Annual Conference on Neural Information Processing Systems, 2017: 6000-6010.
[26] ZHONG Z, CHEN D. A frustratingly easy approach for joint entity and relation extraction [C]//Proceedings of the NAACL, 2021: 50-61.
[27] SOARES L, FITZGERALD N, LING J, et al. Matching the blanks: Distributional similarity for relation learning[C]//Proceedings of the 57th Conference of the Associationfor Computational Linguistics, 2019: 2895-2905.
[28] LIU Y, MENG F, ZHANG J, et al. CM-Net: A novel collaborative memory network for spoken language understanding[C]//Proceedings of the Conferenceon Empirical Methods in Natural Language Processing and the 9th International Joint Conferenceon Natural Language Processing, 2019: 1051-1060.
[29] SRIVASTAVA N, HINTON G, KRIZHEVSKY A, et al. Dropout: A simple way to prevent neural networks from overfitting [J]. Journal of Machine Learning Research, 2014, 15(1): 1929-1958.
[30] KINGMA D, BA J. Adam: A method for stochastic optimization[C]//Proceedings of the 3rd International Conference on Learning Representations, 2015: 1-15.
{{custom_fnGroup.title_cn}}
脚注
{{custom_fn.content}}
基金
广东省自然科学基金(2021A1515011864);广东省智慧农业重点实验室(201902010081);国家自然科学基金(71472068);广东省普通高校特色创新项目(2020KTSCX016);广东省大学生创新训练计划项目(S202010564169,S202110564051)
{{custom_fund}}
PDF(6855 KB)
