Spoken language understanding (SLU) typically includes two subtasks: intent detection and slot filling. Currently, it has achieved great success in high-resource languages, but it still remains challenging in low-resource languages due to the scarcity of labeled training data. Hence, there is a growing interest in zero-shot cross-lingual SLU. Despite of the success of existing zero-shot cross-lingual SLU models, most of them neglect to achieve the mutual guidance between intent and slots. To address this issue, we propose an Intra-Inter Knowledge Distillation framework for zero-shot cross-lingual Spoken Language Understanding (I$^2$KD-SLU) to model the mutual guidance. Specifically, we not only apply intra-knowledge distillation between intent predictions or slot predictions of the same utterance in different languages, but also apply inter-knowledge distillation between intent predictions and slot predictions of the same utterance. Our experimental results demonstrate that our proposed framework significantly improves the performance compared with the strong baselines and achieves the new state-of-the-art performance on the MultiATIS++ dataset, obtaining a significant improvement over the previous best model in overall accuracy.

翻译：口语理解通常包含两个子任务：意图检测与槽位填充。目前，该技术在高资源语言领域已取得显著成功，但在低资源语言中因标注训练数据稀缺仍面临挑战。因此，零样本跨语言口语理解正获得越来越多的关注。尽管现有零样本跨语言口语理解模型已取得一定成效，但多数模型未能实现意图与槽位之间的相互指导。为解决该问题，我们提出了一种面向零样本跨语言口语理解的帧内-帧间知识蒸馏框架（I$^2$KD-SLU），用于建模这种相互指导关系。具体而言，我们不仅对同一语句在不同语言下的意图预测或槽位预测应用帧内知识蒸馏，还对同一语句的意图预测与槽位预测应用帧间知识蒸馏。实验结果表明，与强基线模型相比，我们提出的框架显著提升了性能，并在MultiATIS++数据集上取得了新的最优结果，整体准确率较先前最佳模型实现了显著提升。