Self-supervised learned (SSL) models such as Wav2vec and HuBERT yield state-of-the-art results on speech-related tasks. Given the effectiveness of such models, it is advantageous to use them in conventional ASR systems. While some approaches suggest incorporating these models as a trainable encoder or a learnable frontend, training such systems is extremely slow and requires a lot of computation cycles. In this work, we propose two simple approaches that use (1) framewise addition and (2) cross-attention mechanisms to efficiently incorporate the representations from the SSL model(s) into the ASR architecture, resulting in models that are comparable in size with standard encoder-decoder conformer systems while also avoiding the usage of SSL models during training. Our approach results in faster training and yields significant performance gains on the Librispeech and Tedlium datasets compared to baselines. We further provide detailed analysis and ablation studies that demonstrate the effectiveness of our approach.

翻译：自监督学习模型（如Wav2vec和HuBERT）在语音相关任务中取得了最先进的结果。鉴于此类模型的有效性，将其应用于传统自动语音识别系统具有显著优势。尽管某些方法建议将这些模型作为可训练编码器或可学习前端进行整合，但此类系统的训练速度极慢且需耗费大量计算资源。本研究提出两种简单方法：采用（1）帧级加法与（2）交叉注意力机制，将自监督学习模型的表示高效融入自动语音识别架构，最终模型规模可与标准编码器-解码器Conformer系统相媲美，同时避免在训练过程中使用自监督学习模型。与基线相比，我们的方法在Librispeech和Tedlium数据集上实现了更快的训练速度与显著的性能提升。我们进一步通过详细分析与消融研究验证了该方法的有效性。