Conformer-based models have become the dominant end-to-end architecture for speech processing tasks. With the objective of enhancing the conformer architecture for efficient training and inference, we carefully redesigned Conformer with a novel downsampling schema. The proposed model, named Fast Conformer(FC), is 2.8x faster than the original Conformer, supports scaling to Billion parameters without any changes to the core architecture and also achieves state-of-the-art accuracy on Automatic Speech Recognition benchmarks. To enable transcription of long-form speech up to 11 hours, we replaced global attention with limited context attention post-training, while also improving accuracy through fine-tuning with the addition of a global token. Fast Conformer, when combined with a Transformer decoder also outperforms the original Conformer in accuracy and in speed for Speech Translation and Spoken Language Understanding.

翻译：基于Conformer的架构已成为语音处理任务中主流的端到端结构。为提升Conformer架构在高效训练与推理方面的性能，我们通过引入新颖的下采样方案对其进行了精心重构。所提出的模型名为Fast Conformer（FC），其速度较原始Conformer提升2.8倍，可在不修改核心架构的条件下扩展至十亿参数量级，并在自动语音识别基准测试中达到最先进水平。为实现长达11小时的长语音转录，我们在后训练阶段用有限上下文注意力替代全局注意力，同时通过添加全局标记进行微调以提升准确率。当Fast Conformer与Transformer解码器结合时，在语音翻译和口语理解任务中的准确率与速度均超越原始Conformer。