Automatic Speech Recognition (ASR) models need to be optimized for specific hardware before they can be deployed on devices. This can be done by tuning the model's hyperparameters or exploring variations in its architecture. Re-training and re-validating models after making these changes can be a resource-intensive task. This paper presents TODM (Train Once Deploy Many), a new approach to efficiently train many sizes of hardware-friendly on-device ASR models with comparable GPU-hours to that of a single training job. TODM leverages insights from prior work on Supernet, where Recurrent Neural Network Transducer (RNN-T) models share weights within a Supernet. It reduces layer sizes and widths of the Supernet to obtain subnetworks, making them smaller models suitable for all hardware types. We introduce a novel combination of three techniques to improve the outcomes of the TODM Supernet: adaptive dropouts, an in-place Alpha-divergence knowledge distillation, and the use of ScaledAdam optimizer. We validate our approach by comparing Supernet-trained versus individually tuned Multi-Head State Space Model (MH-SSM) RNN-T using LibriSpeech. Results demonstrate that our TODM Supernet either matches or surpasses the performance of manually tuned models by up to a relative of 3% better in word error rate (WER), while efficiently keeping the cost of training many models at a small constant.

翻译：自动语音识别（ASR）模型在部署至设备端前需针对特定硬件进行优化，可通过调整模型超参数或探索架构变体实现。然而，修改后重新训练与验证模型的过程资源消耗巨大。本文提出TODM（一次训练多次部署）新方法，以等同于单次训练任务的GPU耗时，高效训练出多种尺寸的硬件友好型设备端ASR模型。TODM借鉴了超网络（Supernet）领域的前期研究成果，使循环神经网络变换器（RNN-T）模型在超网络内共享权重。通过缩减超网络的层尺寸与宽度获取子网络，生成适用于各类硬件的小型模型。我们创新性地组合三种技术来提升TODM超网络性能：自适应丢弃法、原地Alpha散度知识蒸馏以及ScaledAdam优化器的应用。通过将超网络训练模型与独立调优的多头状态空间模型（MH-SSM）RNN-T在LibriSpeech数据集上进行对比验证，结果表明：TODM超网络在词错误率（WER）上可达到或超越手动调优模型，最高相对提升3%，同时将多模型训练成本有效控制在恒定低水平。