End-to-end speech recognition generally uses hand-engineered acoustic features as input and excludes the feature extraction module from its joint optimization. To extract learnable and adaptive features and mitigate information loss, we propose a new encoder that adopts globally attentive locally recurrent (GALR) networks and directly takes raw waveform as input. We observe improved ASR performance and robustness by applying GALR on different window lengths to aggregate fine-grain temporal information into multi-scale acoustic features. Experiments are conducted on a benchmark dataset AISHELL-2 and two large-scale Mandarin speech corpus of 5,000 hours and 21,000 hours. With faster speed and comparable model size, our proposed multi-scale GALR waveform encoder achieved consistent character error rate reductions (CERRs) from 7.9% to 28.1% relative over strong baselines, including Conformer and TDNN-Conformer. In particular, our approach demonstrated notable robustness than the traditional handcrafted features and outperformed the baseline MFCC-based TDNN-Conformer model by a 15.2% CERR on a music-mixed real-world speech test set.

翻译：终端到终端语音识别通常使用手工设计的音频特性作为输入,并将地物提取模块排除在联合优化中排除。为了提取可学习和适应性特点并减轻信息损失,我们提议了一个新的编码器,采用全球关注的本地经常性(GALR)网络,直接将原始波形作为输入。我们观察到了ASR的性能和稳健性,在不同的窗口长度上应用GARR将细微时间信息汇总到多尺度的音频特征中。实验是在一个基准数据集AISHELL-2和两个大型曼达林语音资料库上进行的,时间为5 000小时和21 000小时。由于速度更快和类似的模型规模,我们提议的多尺度的GALR波形电波编码器在强大的基线(包括Conexerect和TDNN-Cononfrent)上实现了一致的性格误差率降低,从7.9%降至28.1%。我们的方法比传统手工制作的特征要强得多,而且比以TDCNNN-Conf模型比15.2%的基线模型更强。