It is challenging to extract semantic meanings directly from audio signals in spoken language understanding (SLU), due to the lack of textual information. Popular end-to-end (E2E) SLU models utilize sequence-to-sequence automatic speech recognition (ASR) models to extract textual embeddings as input to infer semantics, which, however, require computationally expensive auto-regressive decoding. In this work, we leverage self-supervised acoustic encoders fine-tuned with Connectionist Temporal Classification (CTC) to extract textual embeddings and use joint CTC and SLU losses for utterance-level SLU tasks. Experiments show that our model achieves 4% absolute improvement over the the state-of-the-art (SOTA) dialogue act classification model on the DSTC2 dataset and 1.3% absolute improvement over the SOTA SLU model on the SLURP dataset.

翻译：口语理解（SLU）中直接从音频信号提取语义含义具有挑战性，这是由于缺乏文本信息。流行的端到端（E2E）SLU模型利用序列到序列自动语音识别（ASR）模型提取文本嵌入作为推断语义的输入，但这需要计算成本高昂的自回归解码。在本研究中，我们利用经连接主义时间分类（CTC）微调的自监督声学编码器提取文本嵌入，并采用联合CTC与SLU损失函数处理语句级别的SLU任务。实验表明，在DSTC2数据集上，我们的模型相较于当前最先进的（SOTA）对话行为分类模型实现了4%的绝对提升，在SLURP数据集上相较于SOTA SLU模型实现了1.3%的绝对提升。