Non-autoregressive (NAR) modeling has gained significant interest in speech processing since these models achieve dramatically lower inference time than autoregressive (AR) models while also achieving good transcription accuracy. Since NAR automatic speech recognition (ASR) models must wait for the completion of the entire utterance before processing, some works explore streaming NAR models based on blockwise attention for low-latency applications. However, streaming NAR models significantly lag in accuracy compared to streaming AR and non-streaming NAR models. To address this, we propose a streaming "semi-autoregressive" ASR model that incorporates the labels emitted in previous blocks as additional context using a Language Model (LM) subnetwork. We also introduce a novel greedy decoding algorithm that addresses insertion and deletion errors near block boundaries while not significantly increasing the inference time. Experiments show that our method outperforms the existing streaming NAR model by 19% relative on Tedlium2, 16%/8% on Librispeech-100 clean/other test sets, and 19%/8% on the Switchboard(SWB) / Callhome(CH) test sets. It also reduced the accuracy gap with streaming AR and non-streaming NAR models while achieving 2.5x lower latency. We also demonstrate that our approach can effectively utilize external text data to pre-train the LM subnetwork to further improve streaming ASR accuracy.

翻译：非自回归（NAR）建模在语音处理领域引起了广泛关注，因为这类模型在实现比自回归（AR）模型显著更低的推理时间的同时，还能保持较高的转录准确率。由于非自回归自动语音识别（ASR）模型必须等待整个语音片段完成之后才能进行处理，一些研究探索了基于分块注意力机制的流式NAR模型，以应用于低延迟场景。然而，流式NAR模型在准确率上明显落后于流式AR模型和非流式NAR模型。为了解决这一问题，我们提出了一种流式"半自回归"ASR模型，该模型通过语言模型（LM）子网络，将先前块中输出的标签作为额外上下文信息融入模型。我们还引入了一种新颖的贪心解码算法，该算法能够有效处理块边界附近的插入和删除错误，同时不显著增加推理时间。实验结果表明，我们的方法在Tedlium2上相对现有流式NAR模型提升了19%，在Librispeech-100的clean/other测试集上分别提升了16%/8%，在Switchboard（SWB）/Callhome（CH）测试集上分别提升了19%/8%。此外，该方法缩小了与流式AR模型和非流式NAR模型之间的准确率差距，同时实现了2.5倍的更低延迟。我们还证明了我们的方法能够有效利用外部文本数据对LM子网络进行预训练，从而进一步提升流式ASR的准确率。