Optimization of modern ASR architectures is among the highest priority tasks since it saves many computational resources for model training and inference. The work proposes a new Uconv-Conformer architecture based on the standard Conformer model. It consistently reduces the input sequence length by 16 times, which results in speeding up the work of the intermediate layers. To solve the convergence issue connected with such a significant reduction of the time dimension, we use upsampling blocks like in the U-Net architecture to ensure the correct CTC loss calculation and stabilize network training. The Uconv-Conformer architecture appears to be not only faster in terms of training and inference speed but also shows better WER compared to the baseline Conformer. Our best Uconv-Conformer model shows 47.8% and 23.5% inference acceleration on the CPU and GPU, respectively. Relative WER reduction is 7.3% and 9.2% on LibriSpeech test_clean and test_other respectively.

翻译：现代ASR架构的优化是最高优先级任务之一，因为它能显著节省模型训练和推理的计算资源。本文提出一种基于标准Conformer模型的新型Uconv-Conformer架构。该架构将输入序列长度一致缩减16倍，从而加速中间层的计算。为解决时间维度大幅缩减带来的收敛问题，我们采用类似U-Net架构的上采样模块，以确保正确的CTC损失计算并稳定网络训练。Uconv-Conformer架构不仅在训练和推理速度上更快，而且在WER指标上优于基线Conformer模型。最佳Uconv-Conformer模型在CPU和GPU上分别实现47.8%和23.5%的推理加速。在LibriSpeech的test_clean和test_other测试集上，相对WER分别降低7.3%和9.2%。