Target-speaker automatic speech recognition (ASR) aims to transcribe the desired speech of a target speaker from multi-talker overlapped utterances. Most of the existing target-speaker ASR (TS-ASR) methods involve either training from scratch or fully fine-tuning a pre-trained model, leading to significant training costs and becoming inapplicable to large foundation models. This work leverages prompt tuning, a parameter-efficient fine-tuning approach, to extend Whisper, a large-scale single-talker ASR model, to TS-ASR. Experimental results show that prompt tuning can achieve performance comparable to state-of-the-art full fine-tuning approaches while only requiring about 1% of task-specific model parameters. Notably, the original Whisper's features, such as inverse text normalization and timestamp prediction, are retained in target-speaker ASR, keeping the generated transcriptions natural and informative.

翻译：目标说话人自动语音识别旨在从多人重叠语音中转录目标说话人的指定语音。现有大多数目标说话人语音识别方法要么从头训练，要么对预训练模型进行全参数微调，这不仅导致高昂的训练成本，而且难以适用于大型基础模型。本研究采用参数高效的微调方法——提示调优，将大规模单说话人语音识别模型Whisper扩展至目标说话人语音识别。实验结果表明，提示调优在仅需约1%任务特定模型参数的情况下，性能即可与最先进的全参数微调方法相媲美。值得注意的是，原始Whisper的特性（如逆文本归一化和时间戳预测）在目标说话人语音识别中得以保留，使得生成的转录结果兼具自然性与信息丰富性。