This paper introduces a novel Token-and-Duration Transducer (TDT) architecture for sequence-to-sequence tasks. TDT extends conventional RNN-Transducer architectures by jointly predicting both a token and its duration, i.e. the number of input frames covered by the emitted token. This is achieved by using a joint network with two outputs which are independently normalized to generate distributions over tokens and durations. During inference, TDT models can skip input frames guided by the predicted duration output, which makes them significantly faster than conventional Transducers which process the encoder output frame by frame. TDT models achieve both better accuracy and significantly faster inference than conventional Transducers on different sequence transduction tasks. TDT models for Speech Recognition achieve better accuracy and up to 2.82X faster inference than RNN-Transducers. TDT models for Speech Translation achieve an absolute gain of over 1 BLEU on the MUST-C test compared with conventional Transducers, and its inference is 2.27X faster. In Speech Intent Classification and Slot Filling tasks, TDT models improve the intent accuracy up to over 1% (absolute) over conventional Transducers, while running up to 1.28X faster.

翻译：本文提出了一种新颖的标记-时长转导器（Token-and-Duration Transducer, TDT）架构，用于序列到序列任务。TDT通过联合预测标记及其时长（即发射标记所覆盖的输入帧数）来扩展传统RNN转导器架构。该实现采用具有两个输出的联合网络，这两个输出分别独立归一化以生成标记和时长的概率分布。在推理过程中，TDT模型可根据预测的时长输出跳过输入帧，从而显著快于逐帧处理编码器输出的传统转导器。实验表明，TDT模型在不同序列转导任务中均实现了比传统转导器更高的准确率和更快的推理速度：在语音识别任务中，TDT模型相比RNN转导器准确率更优且推理速度提升达2.82倍；在语音翻译任务中，TDT模型在MUST-C测试集上比传统转导器获得超过1个BLEU值的绝对增益，推理速度提升2.27倍；在语音意图分类与槽位填充任务中，TDT模型在意图准确率上较传统转导器提升超1%（绝对值），推理速度提升达1.28倍。