Target-speaker speech recognition aims to recognize target-speaker speech from noisy environments with background noise and interfering speakers. This work presents a joint framework that combines time-domain target-speaker speech extraction and Recurrent Neural Network Transducer (RNN-T). To stabilize the joint-training, we propose a multi-stage training strategy that pre-trains and fine-tunes each module in the system before joint-training. Meanwhile, speaker identity and speech enhancement uncertainty measures are proposed to compensate for residual noise and artifacts from the target speech extraction module. Compared to a recognizer fine-tuned with a target speech extraction model, our experiments show that adding the neural uncertainty module significantly reduces 17% relative Character Error Rate (CER) on multi-speaker signals with background noise. The multi-condition experiments indicate that our method can achieve 9% relative performance gain in the noisy condition while maintaining the performance in the clean condition.

翻译：目标演讲者语音识别旨在识别来自吵闹环境、背景噪音和干扰演讲者的声音,这项工作提供了一个联合框架,将时间-主讲人语音提取和经常神经网络转换器(RNN-T)结合起来。为稳定联合培训,我们提议了一个多阶段培训战略,在联合培训之前,在系统每个模块中,先进行编程和微调;同时,还提议了提高演讲者身份和语音增强不确定性措施,以补偿目标语音提取模块中的剩余噪音和艺术品。与一个与目标语音提取模型进行微调的识别器相比,我们的实验显示,增加神经不确定性模块将显著降低17 % 相对于带有背景噪音的多声音信号的相对性差率。多条件实验表明,我们的方法可以在噪音条件下实现9%的相对性能增益,同时保持清洁状态的性能。