Target-speaker automatic speech recognition (ASR) aims to transcribe the desired speech of a target speaker from multi-talker overlapped utterances. Most of the existing target-speaker ASR (TS-ASR) methods involve either training from scratch or fully fine-tuning a pre-trained model, leading to significant training costs and becoming inapplicable to large foundation models. This work leverages prompt tuning, a parameter-efficient fine-tuning approach, to extend Whisper, a large-scale single-talker ASR model, to TS-ASR. Variants of prompt tuning approaches along with their configurations are explored and optimized for TS-ASR.Experimental results show that prompt tuning can achieve performance comparable to state-of-the-art full training approaches while only requiring about 1\% of task-specific model parameters. Notably, the original Whisper's features, such as inverse text normalization and timestamp tagging, are retained in target-speaker ASR, keeping the generated transcriptions natural and informative.

翻译：目标说话人自动语音识别旨在从多说话人重叠语音中转录目标说话人的目标语音。现有的大多数目标说话人语音识别方法要么需要从头训练，要么需要对预训练模型进行全参数微调，导致训练成本高昂，且难以应用于大型基础模型。本研究利用提示调优这一参数高效的微调方法，将大规模单说话人语音识别模型Whisper扩展至目标说话人语音识别。我们探索了多种提示调优方法及其配置，并针对目标说话人语音识别任务进行了优化。实验结果表明，提示调优方法在仅使用约1%任务特定模型参数的情况下，即可达到与全参数训练方法相媲美的性能。值得注意的是，原始Whisper的特性（如反向文本归一化和时间戳标记）在目标说话人语音识别中得以保留，使生成的转录文本既自然又信息丰富。