Recently, end-to-end models have been widely used in automatic speech recognition (ASR) systems. Two of the most representative approaches are connectionist temporal classification (CTC) and attention-based encoder-decoder (AED) models. Autoregressive transformers, variants of AED, adopt an autoregressive mechanism for token generation and thus are relatively slow during inference. In this paper, we present a comprehensive study of a CTC Alignment-based Single-Step Non-Autoregressive Transformer (CASS-NAT) for end-to-end ASR. In CASS-NAT, word embeddings in the autoregressive transformer (AT) are substituted with token-level acoustic embeddings (TAE) that are extracted from encoder outputs with the acoustical boundary information offered by the CTC alignment. TAE can be obtained in parallel, resulting in a parallel generation of output tokens. During training, Viterbi-alignment is used for TAE generation, and multiple training strategies are further explored to improve the word error rate (WER) performance. During inference, an error-based alignment sampling method is investigated in depth to reduce the alignment mismatch in the training and testing processes. Experimental results show that the CASS-NAT has a WER that is close to AT on various ASR tasks, while providing a ~24x inference speedup. With and without self-supervised learning, we achieve new state-of-the-art results for non-autoregressive models on several datasets. We also analyze the behavior of the CASS-NAT decoder to explain why it can perform similarly to AT. We find that TAEs have similar functionality to word embeddings for grammatical structures, which might indicate the possibility of learning some semantic information from TAEs without a language model.

翻译：近来，端到端模型在自动语音识别系统中得到广泛应用。其中最具代表性的两种方法是连接主义时序分类和基于注意力机制的编码器-解码器模型。自回归Transformer作为变体，采用自回归机制生成词元，导致推理速度相对较慢。本文系统研究了基于CTC对齐的单步非自回归Transformer（CASS-NAT）在端到端语音识别中的应用。在CASS-NAT中，自回归Transformer的词嵌入被替换为基于词级的声学嵌入，这些嵌入通过CTC对齐提供的声学边界信息从编码器输出中提取。声学嵌入可并行获得，从而实现输出词元的并行生成。训练阶段使用维特比对齐生成声学嵌入，并进一步探索多种训练策略以降低词错误率。推理阶段深入研究了基于误差的对齐采样方法，以缓解训练与测试过程中的对齐不匹配问题。实验结果表明，CASS-NAT在多种语音识别任务上的词错误率接近自回归Transformer，同时实现约24倍的推理加速。无论是否采用自监督学习，我们在多个数据集上均取得了非自回归模型的最新最优结果。此外，我们通过分析CASS-NAT解码器的行为，解释了其性能接近自回归Transformer的原因。研究发现，声学嵌入在语法结构上具有与词嵌入相似的功能，这暗示了无需语言模型即可从声学嵌入中学习部分语义信息的可能性。