This paper works on non-autoregressive automatic speech recognition. A unimodal aggregation (UMA) is proposed to segment and integrate the feature frames that belong to the same text token, and thus to learn better feature representations for text tokens. The frame-wise features and weights are both derived from an encoder. Then, the feature frames with unimodal weights are integrated and further processed by a decoder. Connectionist temporal classification (CTC) loss is applied for training. Compared to the regular CTC, the proposed method learns better feature representations and shortens the sequence length, resulting in lower recognition error and computational complexity. Experiments on three Mandarin datasets show that UMA demonstrates superior or comparable performance to other advanced non-autoregressive methods, such as self-conditioned CTC. Moreover, by integrating self-conditioned CTC into the proposed framework, the performance can be further noticeably improved.

翻译：本文研究非自回归自动语音识别问题。提出了一种单峰聚合（UMA）方法，用于分割并整合属于同一文本标记的特征帧，从而学习更好的文本标记特征表示。帧级特征及其权重均由编码器推导得出。随后，具备单峰权重的特征帧被整合，并由解码器进一步处理。采用连接主义时序分类（CTC）损失函数进行训练。与常规CTC相比，所提方法能够学习更优的特征表示，并缩短序列长度，从而降低识别错误率和计算复杂度。在三个中文数据集上的实验表明，UMA的性能优于或媲美其他先进非自回归方法（如自条件CTC）。此外，将自条件CTC融入所提框架后，性能可得到进一步显著提升。