We introduce a multilingual speaker change detection model (USM-SCD) that can simultaneously detect speaker turns and perform ASR for 96 languages. This model is adapted from a speech foundation model trained on a large quantity of supervised and unsupervised data, demonstrating the utility of fine-tuning from a large generic foundation model for a downstream task. We analyze the performance of this multilingual speaker change detection model through a series of ablation studies. We show that the USM-SCD model can achieve more than 75% average speaker change detection F1 score across a test set that consists of data from 96 languages. On American English, the USM-SCD model can achieve an 85.8% speaker change detection F1 score across various public and internal test sets, beating the previous monolingual baseline model by 21% relative. We also show that we only need to fine-tune one-quarter of the trainable model parameters to achieve the best model performance. The USM-SCD model exhibits state-of-the-art ASR quality compared with a strong public ASR baseline, making it suitable to handle both tasks with negligible additional computational cost.

翻译：我们提出了一种多语言说话人切换检测模型（USM-SCD），该模型能够同时检测96种语言的说话人切换并执行自动语音识别。该模型从在大规模有监督和无监督数据上训练的语音基础模型微调而来，验证了从大型通用基础模型微调以适配下游任务的实用性。通过一系列消融实验，我们分析了该多语言说话人切换检测模型的性能。结果表明，在包含96种语言数据的测试集上，USM-SCD模型在平均说话人切换检测F1分数上达到75%以上。在美式英语中，USM-SCD模型在多个公开和内部测试集上取得了85.8%的说话人切换检测F1分数，比之前的单语言基线模型相对提升21%。我们还发现，仅需微调可训练模型参数的四分之一即可实现最佳模型性能。与强大的公开语音识别基线相比，USM-SCD模型展现了当前最优的语音识别质量，使其能够以可忽略的额外计算成本同时胜任两项任务。