Convolutional frontends are a typical choice for Transformer-based automatic speech recognition to preprocess the spectrogram, reduce its sequence length, and combine local information in time and frequency similarly. However, the width and height of an audio spectrogram denote different information, e.g., due to reverberation as well as the articulatory system, the time axis has a clear left-to-right dependency. On the contrary, vowels and consonants demonstrate very different patterns and occupy almost disjoint frequency ranges. Therefore, we hypothesize, global attention over frequencies is beneficial over local convolution. We obtain 2.4 % relative word error rate reduction (rWERR) on a production scale Conformer transducer replacing its convolutional neural network frontend by the proposed F-Attention module on Alexa traffic. To demonstrate generalizability, we validate this on public LibriSpeech data with a long short term memory-based listen attend and spell architecture obtaining 4.6 % rWERR and demonstrate robustness to (simulated) noisy conditions.

翻译：摘要：卷积前端是基于Transformer的自动语音识别中处理频谱图、缩短序列长度并类似地融合时间与频率局部信息的典型选择。然而，音频频谱图的宽度和高度表示不同的信息——例如，因混响和发音系统的特性，时间轴具有明确的从左到右依赖关系；相反，元音和辅音表现出截然不同的模式，且占据几乎不重叠的频率范围。因此，我们假设对频率进行全局注意力优于局部卷积。通过在Alexa流量上采用所提出的F-Attention模块替代卷积神经网络前端，我们使生产级Conformer换能器获得了2.4%的相对词错误率降低。为验证泛化能力，我们基于公开LibriSpeech数据，使用基于长短期记忆网络的Listen Attend and Spell架构进行验证，获得了4.6%相对词错误率降低，并展示了其对（模拟）噪声环境的鲁棒性。