多领域口语语言理解包括多意图识别和槽填充两个子任务,现有研究通过构建语句中的意图和槽之间的关联提升模型的表现。然而现有研究将多领域场景下的意图和槽看作相互独立的标签,忽视了标签之间领域内和领域间的结构关联。该文提出细粒度标签图和领域相关图的双图注意力联合模型。具体来说,细粒度标签图将意图和槽标签分成细粒度分片,建模分片之间的结构性关联和上下文表示的语义特征。领域相关图通过标签间的领域信息,建模预测意图和对应领域内槽的关联,减少图中的冗余关联。实验结果表明,在两个公开的数据集上,该文提出的模型均优于基准模型。
Abstract
Multi-domain spoken language understanding consists of two subtasks: multi-intent detection and slot filling. Previous studies have achieved notable performance by establishing correlations between intents and slots, though failing to capture the inherent intra-domain and inter-domain structural correlations between intent and slot labels. This paper introduces a novel joint model based on DualGAT, incorporating the fine-grained label graph and the domain-related graph. The fine-grained label graphs split the intent and slot labels into fine-grained pieces, capturing the structural correlations between the pieces and the semantic features of the contextual representation. The domain-related graph leverages domain information to model correlations between predicted intents and their corresponding slots, thereby reducing redundant correlations in the graph. Experimental results show that our model outperforms the baselines on two publicly available datasets.
关键词
多领域口语语言理解 /
多意图识别 /
细粒度标签图 /
领域相关图
{{custom_keyword}} /
Key words
multi-domain spoken language understanding /
multi-intent detection /
fine-grained label graph /
domain-related graph
{{custom_keyword}} /
{{custom_sec.title}}
{{custom_sec.title}}
{{custom_sec.content}}
参考文献
[1] LOUVAN S, MAGNINI B. Recent neural methods on slot filling and intent classification for task-oriented dialogue systems: A survey[C]//Proceedings of the 28th International Conference on Computational Linguistics, 2020: 480-496.
[2] 杨帆,饶元,丁毅,等.面向任务型的对话系统研究进展[J].中文信息学报,2021,35(10): 1-20.
[3] QIN L, XIE T, CHE W, et al. A survey on spoken language understanding: Recent advances and new frontiers[C]//Proceedings of the 30th International Joint Conference on Artificial Intelligence, 2021: 4577-4584.
[4] LIU B, LANE I. Attention-based recurrent neural network models for joint intent detection and slot filling[C]//Proceedings of the Interspeech, 2016: 685-689.
[5] GOO C, GAO G, HSU Y, et al. Slot-gated modeling for joint slot filling and intent prediction[C]//Proceedings of the Conference of the North American Chapter of the Association for Computational Linguistics, 2018: 753-757.
[6] LI C, LI L, QI J. A self-attentive model with gate mechanism for spoken language understanding[C]//Proceedings of the Conference on Empirical Methods in Natural Language Processing, 2018: 3824-3833.
[7] QIN L, CHE W, LI Y, et al. A stack-propagation framework with token-level intent detection for spoken language understanding[C]//Proceedings of the Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing, 2019: 2078-2087.
[8] E H, NIU P, CHEN Z, et al. A novel bi-directional interrelated model for joint intent detection and slot filling[C]//Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, 2019: 5467-5471.
[9] QIN L, LIU T, CHE W, et al. A co-interactive transformer for joint slot filling and intent detection[C]//Proceedings of the ICASSP IEEE International Conference on Acoustics, Speech and Signal Processing, 2021: 8193-8197.
[10] ZHU Z, HUANG P, HUANG H, et al. A graph attention interactive refine framework with contextual regularization for jointing intent detection and slot filling[C]//Proceedings of the ICASSP IEEE International Conference on Acoustics, Speech and Signal Processing, 2022: 7617-7621.
[11] GANGADHARAIAH R, NARAYANASWAMY B. Joint multiple intent detection and slot labeling for goal-oriented dialog[C]//Proceedings of the Conference of the North American Chapter of the Association for Computational Linguistics, 2019: 564-569.
[12] VELICKOVIC P, CUCURULL G, CASANOVA A, et al. Graph attention networks[C]//Proceedings of the International Conference on Learning Representations, 2018.
[13] QIN L, XU X, CHE W, et al. AGIF: An adaptive graph-interactive framework for joint multiple intent detection and slot filling[C]//Proceedings of the Findings of the Association for Computational Linguistics: EMNLP, 2020: 1807-1816.
[14] QIN L, WEI F, XIE T, et al. GL-GIN: Fast and accurate non-autoregressive model for joint multiple intent detection and slot filling[C]//Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing, 2021: 178-188.
[15] ZHANG X, WANG H. A joint model of intent determination and slot filling for spoken language understanding[C]//Proceedings of the 25th International Joint Conference on Artificial Intelligence, 2016: 2993-2999.
[16] 马天宇,覃俊,刘晶,等.基于BERT的意图分类与槽填充联合方法[J].中文信息学报,2022,36(08): 127-134.
[17] TENG D, QIN L, CHE W, et al. Injecting word information with multi-level word adapter for Chinese spoken language understanding[C]//Proceedings of the ICASSP IEEE International Conference on Acoustics, Speech and Signal Processing, 2021: 8188-8192.
[18] 朱展标,黄沛杰,张业兴,等.中文口语语言理解中依赖引导的字特征槽填充模型[J].中文信息学报,2022,36(08): 118-126.
[19] JIA X, PAN J, YUAN Y, et al. Multi-intent attention and top-k network with interactive framework for joint multiple intent detection and slot filling[C]//Proceedings of the Natural Language Processing and Chinese Computing: 10th CCF International Conference, 2021: 467-479.
[20] VASWANI A, SHAZEER N, PARMAR N, et al. Attention is all you need[C]//Proceedings of the Advances in Neural Information Processing Systems, 2017: 5998-6008.
[21] ZHU Q, HUANG K, ZHANG Z, et al. Crosswoz: A large-scale Chinese cross-domain task-oriented dialogue dataset[J]. Transactions of the Association for Computational Linguistics, 2020, 8: 281-295.
[22] QUAN J, ZHANG S, CAO Q, et al. RiSAWOZ: A large-scale multi-domain wizard-of-oz dataset with rich semantic annotations for task-oriented dialogue modeling[C]//Proceedings of the Conference on Empirical Methods in Natural Language Processing, 2020: 930-940.
[23] CUI Y, CHE W, LIU T, et al. Pre-training with whole word masking for Chinese bert[J]. IEEE/ACM Transactions on Audio, Speech, and Language Processing, 2021, 29: 3504-3514.
[24] PAN W, CHEN Q, XU X, et al. A preliminary evaluation of chatgpt for zero-shot dialogue understanding[J/OL]. arXiv preprint arXiv: 2304.04256, 2023.
{{custom_fnGroup.title_cn}}
脚注
{{custom_fn.content}}
基金
科技创新2030-“新一代人工智能”重大项目(2021ZD0113304);国家自然科学基金(62072346);湖北省重点研发计划项目(2021BBA099、2021BBA029)
{{custom_fund}}
PDF(2212 KB)
