Although Large Language Models(LLMs) can generate coherent and contextually relevant text, they often struggle to recognise the intent behind the human user's query. Natural Language Understanding (NLU) models, however, interpret the purpose and key information of user's input to enable responsive interactions. Existing NLU models generally map individual utterances to a dual-level semantic frame, involving sentence-level intent and word-level slot labels. However, real-life conversations primarily consist of multi-turn conversations, involving the interpretation of complex and extended dialogues. Researchers encounter challenges addressing all facets of multi-turn dialogue conversations using a unified single NLU model. This paper introduces a novel approach, MIDAS, leveraging a multi-level intent, domain, and slot knowledge distillation for multi-turn NLU. To achieve this, we construct distinct teachers for varying levels of conversation knowledge, namely, sentence-level intent detection, word-level slot filling, and conversation-level domain classification. These teachers are then fine-tuned to acquire specific knowledge of their designated levels. A multi-teacher loss is proposed to facilitate the combination of these multi-level teachers, guiding a student model in multi-turn dialogue tasks. The experimental results demonstrate the efficacy of our model in improving the overall multi-turn conversation understanding, showcasing the potential for advancements in NLU models through the incorporation of multi-level dialogue knowledge distillation techniques.

翻译：尽管大型语言模型（LLM）能够生成连贯且上下文相关的文本，但其在识别用户查询背后的意图方面仍存在困难。自然语言理解（NLU）模型则通过解析用户输入的目的与关键信息，以实现响应式交互。现有的NLU模型通常将单个语句映射至包含句子级意图与词级槽位标签的双层语义框架。然而，现实对话主要由多轮对话构成，涉及对复杂长程对话的理解。研究者在使用统一单NLU模型处理多轮对话的各方面时面临挑战。本文提出一种新方法MIDAS，通过多层级意图、领域与槽位知识蒸馏实现多轮NLU。为此，我们针对不同层级的对话知识构建了独立的教师模型，包括句子级意图检测、词级槽位填充与会话级领域分类。这些教师模型经过微调以获取其对应层级的特定知识。我们提出一种多教师损失函数，以促进多层级教师模型的协同整合，从而在多轮对话任务中指导学生模型。实验结果表明，该模型能有效提升整体多轮对话理解能力，展现了通过引入多层级对话知识蒸馏技术推动NLU模型发展的潜力。