Modeling the errors of a speech recognizer can help simulate errorful recognized speech data from plain text, which has proven useful for tasks like discriminative language modeling, improving robustness of NLP systems, where limited or even no audio data is available at train time. Previous work typically considered replicating behavior of GMM-HMM based systems, but the behavior of more modern posterior-based neural network acoustic models is not the same and requires adjustments to the error prediction model. In this work, we extend a prior phonetic confusion based model for predicting speech recognition errors in two ways: first, we introduce a sampling-based paradigm that better simulates the behavior of a posterior-based acoustic model. Second, we investigate replacing the confusion matrix with a sequence-to-sequence model in order to introduce context dependency into the prediction. We evaluate the error predictors in two ways: first by predicting the errors made by a Switchboard ASR system on unseen data (Fisher), and then using that same predictor to estimate the behavior of an unrelated cloud-based ASR system on a novel task. Sampling greatly improves predictive accuracy within a 100-guess paradigm, while the sequence model performs similarly to the confusion matrix.

翻译：建模语音识别器的错误有助于从纯文本模拟含有错误的识别语音数据，这在训练阶段音频数据有限甚至完全缺失的情况下，已被证明对区分性语言建模、提升自然语言处理系统鲁棒性等任务具有实用价值。先前研究通常侧重于复现基于GMM-HMM系统的行为，但基于后验概率的现代神经网络声学模型具有不同特性，需要对错误预测模型进行调整。本研究从两个方面扩展了先前基于音素混淆的语音识别错误预测模型：首先，我们引入基于采样的范式以更好地模拟后验概率声学模型的行为；其次，我们探索用序列到序列模型替代混淆矩阵，从而将上下文依赖性引入预测过程。我们通过两种方式评估错误预测器：首先预测Switchboard ASR系统在未见数据（Fisher）上的错误，随后使用同一预测器估计不相关的云端ASR系统在新任务上的行为。在100次猜测的评估范式下，采样方法显著提升了预测准确率，而序列模型的表现与混淆矩阵相当。