Pre-training speech models on large volumes of data has achieved remarkable success. OpenAI Whisper is a multilingual multitask model trained on 680k hours of supervised speech data. It generalizes well to various speech recognition and translation benchmarks even in a zero-shot setup. However, the full pipeline for developing such models (from data collection to training) is not publicly accessible, which makes it difficult for researchers to further improve its performance and address training-related issues such as efficiency, robustness, fairness, and bias. This work presents an Open Whisper-style Speech Model (OWSM), which reproduces Whisper-style training using an open-source toolkit and publicly available data. OWSM even supports more translation directions and can be more efficient to train. We will publicly release all scripts used for data preparation, training, inference, and scoring as well as pre-trained models and training logs to promote open science.

翻译：在大规模数据上预训练语音模型已取得显著成功。OpenAI Whisper是一个在68万小时监督语音数据上训练的多语言多任务模型，即使在零样本设置下也能很好地泛化到各种语音识别和翻译基准测试。然而，开发此类模型的完整流程（从数据收集到训练）并未公开，这使得研究人员难以进一步改进其性能，并解决与训练相关的效率、鲁棒性、公平性和偏差等问题。本文提出了开放式Whisper风格语音模型（OWSM），该模型使用开源工具包和公开数据复现了Whisper风格的训练。OWSM甚至支持更多翻译方向，并且训练效率更高。我们将公开发布所有用于数据准备、训练、推理和评分的脚本，以及预训练模型和训练日志，以促进开放科学。