We introduce a multilingual speaker change detection model (USM-SCD) that can simultaneously detect speaker turns and perform ASR for 96 languages. This model is adapted from a speech foundation model trained on a large quantity of supervised and unsupervised data, demonstrating the utility of fine-tuning from a large generic foundation model for a downstream task. We analyze the performance of this multilingual speaker change detection model through a series of ablation studies. We show that the USM-SCD model can achieve more than 75% average speaker change detection F1 score across a test set that consists of data from 96 languages. On American English, the USM-SCD model can achieve an 85.8% speaker change detection F1 score across various public and internal test sets, beating the previous monolingual baseline model by 21% relative. We also show that we only need to fine-tune one-quarter of the trainable model parameters to achieve the best model performance. The USM-SCD model exhibits state-of-the-art ASR quality compared with a strong public ASR baseline, making it suitable to handle both tasks with negligible additional computational cost.

翻译：我们提出了一种多语种说话人变化检测模型（USM-SCD），该模型能够同时检测96种语言的说话人转换并执行自动语音识别（ASR）。该模型基于一个使用大量有监督和无监督数据训练的语音基础模型进行微调，展示了从大型通用基础模型微调至下游任务的实用价值。通过一系列消融实验，我们分析了该多语种说话人变化检测模型的性能。研究表明，在包含96种语言数据的测试集上，USM-SCD模型的平均说话人变化检测F1分数超过75%。针对美式英语，USM-SCD模型在多个公开和内部测试集上实现了85.8%的说话人变化检测F1分数，较之前的单语基线模型相对提升21%。我们还发现，仅需微调四分之一的可训练模型参数即可达到最佳性能。与强大的公开ASR基线相比，USM-SCD模型展现出最先进的ASR质量，使其能够在几乎不增加额外计算成本的情况下处理这两类任务。