In embedding-matching acoustic-to-word (A2W) ASR, every word in the vocabulary is represented by a fixed-dimension embedding vector that can be added or removed independently of the rest of the system. The approach is potentially an elegant solution for the dynamic out-of-vocabulary (OOV) words problem, where speaker- and context-dependent named entities like contact names must be incorporated into the ASR on-the-fly for every speech utterance at testing time. Challenges still remain, however, in improving the overall accuracy of embedding-matching A2W. In this paper, we contribute two methods that improve the accuracy of embedding-matching A2W. First, we propose internally producing multiple embeddings, instead of a single embedding, at each instance in time, which allows the A2W model to propose a richer set of hypotheses over multiple time segments in the audio. Second, we propose using word pronunciation embeddings rather than word orthography embeddings to reduce ambiguities introduced by words that have more than one sound. We show that the above ideas give significant accuracy improvement, with the same training data and nearly identical model size, in scenarios where dynamic OOV words play a crucial role. On a dataset of queries to a speech-based digital assistant that include many user-dependent contact names, we observe up to 18% decrease in word error rate using the proposed improvements.

翻译：在嵌入匹配声学-到-词（A2W）自动语音识别中，词汇表中的每个单词由固定维度的嵌入向量表示，该向量可独立于系统其余部分进行添加或移除。该方法为动态词汇外（OOV）词问题提供了潜在的优雅解决方案——在测试时，需要针对每句语音实时将联系人名称等依赖说话人和上下文的命名实体纳入ASR系统。然而，在提升嵌入匹配A2W整体准确率方面仍面临挑战。本文提出两种改进嵌入匹配A2W准确率的方法。首先，我们提出在每一时刻内部生成多个嵌入而非单一嵌入，使A2W模型能在音频中多个时间片段上提出更丰富的假设集。其次，我们提出使用单词发音嵌入而非单词正字法嵌入，以减少由多音词引入的歧义。实验表明，在动态OOV词起关键作用的场景中，上述方法在使用相同训练数据且模型规模几乎一致的情况下，显著提升了准确率。在包含大量用户依赖联系人名称的语音数字助手查询数据集上，采用所提改进方法后，词错误率降低了高达18%。