Speech-to-text errors made by automatic speech recognition (ASR) system negatively impact downstream models relying on ASR transcriptions. Language error correction models as a post-processing text editing approach have been recently developed for refining the source sentences. However, efficient models for correcting errors in ASR transcriptions that meet the low latency requirements of industrial grade production systems have not been well studied. In this work, we propose a novel non-autoregressive (NAR) error correction approach to improve the transcription quality by reducing word error rate (WER) and achieve robust performance across different upstream ASR systems. Our approach augments the text encoding of the Transformer model with a phoneme encoder that embeds pronunciation information. The representations from phoneme encoder and text encoder are combined via multi-modal fusion before feeding into the length tagging predictor for predicting target sequence lengths. The joint encoders also provide inputs to the attention mechanism in the NAR decoder. We experiment on 3 open-source ASR systems with varying speech-to-text transcription quality and their erroneous transcriptions on 2 public English corpus datasets. Results show that our PATCorrect (Phoneme Augmented Transformer for ASR error Correction) consistently outperforms state-of-the-art NAR error correction method on English corpus across different upstream ASR systems. For example, PATCorrect achieves 11.62% WER reduction (WERR) averaged on 3 ASR systems compared to 9.46% WERR achieved by other method using text only modality and also achieves an inference latency comparable to other NAR models at tens of millisecond scale, especially on GPU hardware, while still being 4.2 - 6.7x times faster than autoregressive models on Common Voice and LibriSpeech datasets.

翻译：摘要：自动语音识别（ASR）系统产生的语音转文本错误会对依赖ASR转录的下游模型产生负面影响。语言纠错模型作为后处理文本编辑方法，近期被用于优化源句。然而，针对ASR转录错误的高效校正模型尚未得到充分研究，这类模型需满足工业级生产系统的低延迟要求。本文提出一种新颖的非自回归（NAR）错误校正方法，通过降低词错误率（WER）提升转录质量，并在不同上游ASR系统中实现鲁棒性能。该方法利用嵌入发音信息的音素编码器增强Transformer模型的文本编码。音素编码器与文本编码器的表征通过多模态融合方式结合，随后输入长度标签预测器以预测目标序列长度。联合编码器还为NAR解码器中的注意力机制提供输入。我们在3种具有不同语音转文本质量的开源ASR系统上，使用其针对2个公开英文语料库数据集生成的错误转录进行实验。结果表明，我们的PATCorrect（面向ASR错误校正的音素增强Transformer）在不同上游ASR系统的英文语料上均持续优于最先进的NAR错误校正方法。例如，PATCorrect在3种ASR系统上平均实现11.62%的WER降低（WERR），而使用纯文本模态的对比方法仅达到9.46%的WERR；同时其推理延迟与其他NAR模型相当（数十毫秒级，尤其在GPU硬件上），且在Common Voice和LibriSpeech数据集上仍比自回归模型快4.2-6.7倍。