We introduce a multilingual speaker change detection model (USM-SCD) that can simultaneously detect speaker turns and perform ASR for 96 languages. This model is adapted from a speech foundation model trained on a large quantity of supervised and unsupervised data, demonstrating the utility of fine-tuning from a large generic foundation model for a downstream task. We analyze the performance of this multilingual speaker change detection model through a series of ablation studies. We show that the USM-SCD model can achieve more than 75% average speaker change detection F1 score across a test set that consists of data from 96 languages. On American English, the USM-SCD model can achieve an 85.8% speaker change detection F1 score across various public and internal test sets, beating the previous monolingual baseline model by 21% relative. We also show that we only need to fine-tune one-quarter of the trainable model parameters to achieve the best model performance. The USM-SCD model exhibits state-of-the-art ASR quality compared with a strong public ASR baseline, making it suitable to handle both tasks with negligible additional computational cost.

翻译：摘要：我们提出一种多语种说话者切换检测模型（USM-SCD），该模型可同时检测96种语言的说话者转换并执行自动语音识别（ASR）。该模型基于在大规模有监督和无监督数据上训练的语音基础模型进行适配，证明了从大型通用基础模型微调以应对下游任务的实用性。通过一系列消融研究，我们分析了该多语种说话者切换检测模型的性能。研究表明，在包含96种语言数据的测试集上，USM-SCD模型的平均说话者切换检测F1分数超过75%。针对美式英语，该模型在多种公开及内部测试集上达到85.8%的说话者切换检测F1分数，相较之前的单语基线模型相对提升21%。此外，我们仅需微调四分之一的可训练模型参数即可实现最佳性能。与强公开ASR基线相比，USM-SCD模型展现出最先进的ASR质量，使其能够以可忽略的额外计算成本同时处理两项任务。