We study speech intent classification and slot filling (SICSF) by proposing to use an encoder pretrained on speech recognition (ASR) to initialize an end-to-end (E2E) Conformer-Transformer model, which achieves the new state-of-the-art results on the SLURP dataset, with 90.14% intent accuracy and 82.27% SLURP-F1. We compare our model with encoders pretrained on self-supervised learning (SSL), and show that ASR pretraining is much more effective than SSL for SICSF. To explore parameter efficiency, we freeze the encoder and add Adapter modules, and show that parameter efficiency is only achievable with an ASR-pretrained encoder, while the SSL encoder needs full finetuning to achieve comparable results. In addition, we provide an in-depth comparison on end-to-end models versus cascading models (ASR+NLU), and show that E2E models are better than cascaded models unless an oracle ASR model is provided. Last but not least, our model is the first E2E model that achieves the same performance as cascading models with oracle ASR. Code, checkpoints and configs are available.

翻译：我们研究语音意图分类与槽填充（SICSF）任务，提出采用在语音识别（ASR）任务上预训练的编码器初始化端到端（E2E）Conformer-Transformer模型。该模型在SLURP数据集上取得了最新的最优结果，意图准确率达90.14%，SLURP-F1分数达82.27%。我们将模型与基于自监督学习（SSL）预训练的编码器进行对比，证明ASR预训练对SICSF任务的效果远优于SSL预训练。为探索参数效率，我们冻结编码器并添加适配器模块，实验表明仅ASR预训练编码器可实现参数高效，而SSL编码器需完整微调才能达到同等性能。此外，我们深入对比了端到端模型与级联模型（ASR+NLU），证明除非配备理想型ASR模型，否则E2E模型性能优于级联模型。最后，本模型是首个在理想ASR条件下达到与级联模型相同性能的E2E模型。相关代码、检查点及配置文件均已开放。