中文口语语言理解中依赖引导的字特征槽填充模型

朱展标,黄沛杰,张业兴,刘树东,张华林,黄均曜,林丕源

PDF(2130 KB)
PDF(2130 KB)
中文信息学报 ›› 2022, Vol. 36 ›› Issue (8) : 118-126.
自然语言理解与生成

中文口语语言理解中依赖引导的字特征槽填充模型

  • 朱展标1,黄沛杰1,2,张业兴1,刘树东1,张华林1,黄均曜1,林丕源1,2
作者信息 +

A Dependency-Guided Character-Based Slot Filling Model for Chinese Spoken Language Understanding

  • ZHU Zhanbiao1, HUANG Peijie1,2, ZHANG Yexing1, LIU Shudong1, ZHANG Hualin1, HUANG Junyao1, LIN Piyuan1,2
Author information +
History +

摘要

意图识别和槽信息填充的联合模型将口语语言理解(Spoken Language Understanding,SLU)技术提升到了一个新的水平,但由于存在出现频率低或未见过的槽指称项(0-shot slot mentions),模型的序列标注性能受限,而且这些联合模型往往没有利用输入序列存在的语法知识信息。已有研究表明,序列标注任务可以通过引入依赖树结构,辅助推断序列标注中槽的存在。由于中文话语由一串字序列组成,在中文口语语言理解中,输入话语的字和槽信息是一一对应的,因而槽信息填充模型往往是字特征模型。基于词的依赖树结构无法直接应用于基于字特征的槽填充模型。为了解决字词之间的矛盾,该文提出了一种基于字模型的依赖引导槽填充模型(Dependency-guided Character-based Slot Filling model,DCSF),提供一种简洁的方法用于解决将词级依赖树结构引入中文字特征模型的冲突,同时通过对话语中词汇内部关系进行建模,保留了词级上下文信息和分词信息。在公共基准语料库SMP-ECDT和CrossWOZ上的实验结果表明,该模型优于比较模型,特别是在未见过的槽指称项和低资源情况下有很大的改进。

Abstract

The joint model for intent detection and slot filling have boosted the state of the art of spoken language understanding (SLU). However, the presence of rarely seen or unseen mention degrades the performance of the model. Earlier researches show that sequence labeling task can benefit from the use of dependency tree structure for inferring existence of slot tags. In Chinese spoken language understanding, dominant models for slot filling are character-based hence word-level dependency tree structure can't be integrated into model directly. In this paper, we propose a dependency-guided character-based slot filling (DCSF) model, which provides a concise way to resolve the conflict of incorporating the word-level dependency tree structure into the character-level model in Chinese. Our DCSF model can integrate dependency tree information into the character-level model while preserving word-level context and segmentation information by modeling different types of relationships between Chinese characters in the utterance. Experimental results on the public benchmark corpora SMP-ECDT and CrossWOZ show our model outperforms the compared models and has a great improvement, especially in low resource and unseen slot mentions scenario.

关键词

口语对话理解 / 槽信息填充 / 依赖结构 / 字特征模型

Key words

spoken language understanding / slot filling / dependency structure / character-based model

引用本文

导出引用
朱展标,黄沛杰,张业兴,刘树东,张华林,黄均曜,林丕源. 中文口语语言理解中依赖引导的字特征槽填充模型. 中文信息学报. 2022, 36(8): 118-126
ZHU Zhanbiao, HUANG Peijie, ZHANG Yexing, LIU Shudong, ZHANG Hualin, HUANG Junyao, LIN Piyuan. A Dependency-Guided Character-Based Slot Filling Model for Chinese Spoken Language Understanding. Journal of Chinese Information Processing. 2022, 36(8): 118-126

参考文献

[1] Tür G, Mori R. Spoken language understanding: systems for extracting semantic information from speech[M]. John Wiley and Sons, Inc.,2011.
[2] Liu B, Lane I. Attention-based recurrent neural network models for joint intent detection and slot filling[C]//Proceedings of the 17th Annual Conference of the International Speech Communication Association, Interspeech 2016: 685-689.
[3] Yao K, Peng B, Zhang Y, et al. Spoken language understanding using long short-term memory neural networks[C]//Proceedings of IEEE Spoken Language Technology Workshop, 2014: 189-194.
[4] Li C, Li L, Qi J. A self-attentive model with gate mechanism for spoken language understanding [C]//Proceedings of the Conference on Empirical Methods in Natural Language Processing, 2018: 3824-3833.
[5] Goo C, Gao G, Y Hsu, et al. Slot-gated modeling for joint slot filling and intent prediction[C]//Proceedings of the Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, 2018: 753-757.
[6] H E, Niu P, Chen Z, et al. A novel bi-directional interrelated model for joint intent detection and slot filling[C]//Proceedings of the 57th Conference of the Association for Computational Linguistics, 2019: 5467-5471.
[7] Qin L, Che W, Li Y, et al. A stack-propagation framework with token-level intent detection for spoken language understanding[C]//Proceedings of the Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing, 2019: 2078-2087.
[8] Hakkani-Tür D, Tür G, elikyilmaz A, et al. Multi-domain joint semantic frame parsing using bi-directional RNN-LSTM[C]//Proceedings of the 17th Annual Conference of the International Speech Communication Association, 2016: 715-719.
[9] Chen Q, Zhuo Z, Wang W. BERT for joint intent classification and slot filling[OL] CoRR, abs/1902.10909, 2019.
[10] Zhang C, Li Y, Du N,et al. Joint slot filling and intent detection via capsule neural networks[C]//Proceedings of the 57th Conference of the Association for Computational Linguistics, 2019: 5259-5267.
[11] Huang L, Sil A, Ji H, et al. Improving slot filling performance with attentive neural networks on dependency structures[C]//Proceedings of the Conference on Empirical Methods in Natural Language Processing, 2017: 2588-2597.
[12] Jie Z, Lu W. Dependency-guided LSTM-CRF for named entity recognition[C]//Proceedings of the Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing, 2019: 3860-3870.
[13] Nie Y, Tian Y, Song Y, et al. Improving named entity recognition with attentive ensemble of syntactic information[C]//Proceedings of the Conference on Empirical Methods in Natural Language Processing: Findings, 2020: 4231-4245.
[14] Zhang Y, Yang J. Chinese NER using lattice LSTM[C]//Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics, 2018: 1554-1564.
[15] Devlin J, Chang M, Lee K, et al. BERT: Pre-training of deep bidirectional transformers for language understanding[C]//Proceedings of the Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, 2019: 4171-4186.
[16] 俞凯,陈露,陈博,等. 任务型人机对话系统中的认知技术: 概念、进展及其未来[J]. 计算机学报, 2015, 38(12): 2333-2348.
[17] 赵阳洋,王振宇,王佩,等. 任务型对话系统研究综述[J]. 计算机学报, 2020, 43(10): 1862-1896.
[18] Haffner P, Tür G, Wright J H. Optimizing svms for complex call classification[C]//Proceedings of the IEEE International Conference on Acoustics, Speech, and Signal Processing, 2003: 632-635.
[19] ESchapire R, Singer Y. Boostexter: A boosting-based system for text categorization[J]. Mach. Learn., 2000, 39(2/3): 135-168.
[20] Raymond C, Riccardi G. Generative and discriminative algorithms for spoken language understanding[C]//Proceedings of the 8th Anual Conference of the International Speech Communication Asociation, 2007: 1605-1608.
[21] Kim Y. Convolutional neural networks for sentence classification[C]//Proceedings of the Conference on Empirical Methods in Natural Language Processing, 2014: 1746-1751.
[22] Mesnil G, Dauphin Y N, Yao K, et al. Using recurrent neural networks for slot filling in spoken language understanding[J]. IEEE ACM Trans. Audio Speech Lang. Process., 2015, 23(3): 530-539.
[23] Yao K, Peng B, Zweig G, et al. Recurrent conditional random field for languageunderstanding[C]//Proceedings of the 39th IEEE International Conference on Acoustics, Speech and Signal Processing, 2014: 4077-4081.
[24] Zhang X, Wang H. A joint model of intent determination and slot filling for spoken language understanding[C]//Proceedings of the 25th International Joint Conference on Artificial Intelligence, 2016: 2993-2999.
[25] Ling X, Weld D S. Fine grained entity recognition[C]//Proceedings of the 26th AAAI Conference on Artificial Intelligence, 2012: 94-100.
[26] Liu J, Huang M, Zhu X. Recognizing biomedical named entities using skip-chain conditional random fields[C]//Proceedings of the Workshop on Biomedical Natural Language Processing, 2010: 10-18.
[27] Liu W, Xu T, Xu Q, et al. An encoding strategy based word-character LSTM for Chinese NER[C]//Proceedings of the Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, 2019: 2379-2389.
[28] Hochreiter S, Schmidhuber J. Long short-term memory[J].Neural Comput, 1997, 9(8): 1735-1780.
[29] Zhu Q, Huang K, Zhang Z, et al. CrossWOZ: A large-scale Chinese cross-domain task-oriented dialogue dataset. Trans. Assoc[J].Comput. Linguistics, 2020, 8: 281-295.
[30] Manning C D, Surdeanu M, Bauer J, et al. The Stanford CoreNLP natural language processing toolkit [C]//Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics, 2014: 55-60.
[31] Srivastava N, Hinton G E, Krizhevsky A, et al. Dropout: A simple way to prevent neural networks from overfitting[J]J. Mach. Learn. Res, 2014, 15(1): 1929-1958.
[32] PKingma D, Ba J. Adam: A method for stochastic optimization[C]//Proceedings of the 3rd International Conference on Learning Representations, 2015.
[33] Ma R, Peng M, Zhang Q, et al. Simplify the usage of lexicon in Chinese NER[C]//Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, 2020: 5951-5960.

基金

广东省自然科学基金(2021A1515011864);广东省智慧农业重点实验室(201902010081);国家自然科学基金(71472068);广东省普通高校特色创新项目(2020KTSCX016);广东省大学生创新训练计划项目(S202010564169,S202110564051)
PDF(2130 KB)

Accesses

Citation

Detail

段落导航
相关文章

/