Slot filling is a crucial subtask in spoken language understanding (SLU), traditionally implemented as a cascade of speech recognition followed by one or more natural language understanding (NLU) components. The recent advent of speech-based large language models (speechLLMs), which integrate speech and textual foundation models, has opened new avenues for achieving speech understanding tasks in a more unified, generative, and instruction-following manner while promising data and compute efficiency with zero-shot abilities, generalizing to unseen slot labels. We address the slot-filling task by creating an empirical upper bound for the task, identifying performance, robustness, and generalization gaps, and proposing improvements to the training data, architecture, and training strategies to narrow the gap with the upper bound result. We show that each of these measures improve performance substantially, while highlighting practical challenges and providing empirical guidance and insights for harnessing these emerging models.

翻译：槽填充是口语理解中的关键子任务，传统上通过语音识别与一个或多个自然语言理解组件的级联实现。近期基于语音的大语言模型的出现，通过整合语音与文本基础模型，为以更统一、生成式和指令遵循的方式实现语音理解任务开辟了新途径，同时凭借零样本能力在数据和计算效率方面展现出潜力，能够泛化至未见过的槽标签。我们通过建立该任务的实证性能上限、识别性能差距与鲁棒性及泛化缺陷，并从训练数据、模型架构和训练策略三方面提出改进方案以缩小与上限结果的差距，从而系统性地解决槽填充任务。研究表明各项改进措施均能显著提升性能，同时揭示了实际应用中的挑战，并为利用这类新兴模型提供了实证指导与见解。