Reliable slot and intent detection (SID) is crucial in natural language understanding for applications like digital assistants. Encoder-only transformer models fine-tuned on high-resource languages generally perform well on SID. However, they struggle with dialectal data, where no standardized form exists and training data is scarce and costly to produce. We explore zero-shot transfer learning for SID, focusing on multiple Bavarian dialects, for which we release a new dataset for the Munich dialect. We evaluate models trained on auxiliary tasks in Bavarian, and compare joint multi-task learning with intermediate-task training. We also compare three types of auxiliary tasks: token-level syntactic tasks, named entity recognition (NER), and language modelling. We find that the included auxiliary tasks have a more positive effect on slot filling than intent classification (with NER having the most positive effect), and that intermediate-task training yields more consistent performance gains. Our best-performing approach improves intent classification performance on Bavarian dialects by 5.1 and slot filling F1 by 8.4 percentage points.

翻译：可靠的槽位与意图检测（SID）对于数字助理等自然语言理解应用至关重要。基于高资源语言微调的仅编码器Transformer模型通常在SID任务上表现良好。然而，这些模型在处理方言数据时面临挑战，因为方言缺乏标准化形式且训练数据稀缺、标注成本高昂。本研究探索了SID的零样本迁移学习方法，重点关注多种巴伐利亚方言，并为此发布了慕尼黑方言的新数据集。我们评估了通过巴伐利亚语辅助任务训练的模型，比较了联合多任务学习与中间任务训练策略。同时对比了三类辅助任务：词元级句法任务、命名实体识别（NER）以及语言建模。研究发现，所采用的辅助任务对槽位填充的积极影响大于意图分类（其中NER的促进作用最为显著），且中间任务训练能带来更稳定的性能提升。我们提出的最佳方法将巴伐利亚方言的意图分类准确率提升了5.1个百分点，槽位填充F1值提高了8.4个百分点。