Conformer-based models have become the dominant end-to-end architecture for speech processing tasks. With the objective of enhancing the conformer architecture for efficient training and inference, we carefully redesigned Conformer with a novel downsampling schema. The proposed model, named Fast Conformer(FC), is 2.8x faster than the original Conformer, supports scaling to Billion parameters without any changes to the core architecture and also achieves state-of-the-art accuracy on Automatic Speech Recognition benchmarks. To enable transcription of long-form speech up to 11 hours, we replaced global attention with limited context attention post-training, while also improving accuracy through fine-tuning with the addition of a global token. Fast Conformer, when combined with a Transformer decoder also outperforms the original Conformer in accuracy and in speed for Speech Translation and Spoken Language Understanding.

翻译：基于 Conformer 的模型已成为语音处理任务中占主导地位的端到端架构。为了提高 Conformer 架构在高效训练和推理方面的性能，我们精心设计了一种新的下采样方案，重新设计了 Conformer。提出的模型名为 Fast Conformer（FC），其速度比原始 Conformer 快 2.8 倍，支持扩展到十亿参数而无需更改核心架构，并在自动语音识别基准测试中实现了最先进的准确率。为了实现对长达 11 小时的长语音转录，我们在训练后使用有限上下文注意力替换了全局注意力，同时通过添加全局令牌进行微调来提高准确率。Fast Conformer 在与 Transformer 解码器结合时，在语音翻译和口语理解方面，其准确率和速度均优于原始 Conformer。