We present a systematic study of fine-tuning OpenAI's Whisper large-v3 for Swiss German ASR, using 1,367 hours of broadcast speech paired with Standard German subtitles as weak supervision. Through 16 iterative training runs on an NVIDIA DGX Spark (Grace Blackwell, 128 GB unified memory, up to 1 PFLOP FP4), we compare LoRA and full fine-tuning of the 1.55B-parameter model, investigate hallucination root causes, and quantify the effect of data quality, subtitle alignment, and training strategy. Our best model achieves 25.6% measured WER on the All Swiss German Dialects Test Set (ASGDTS) in an honest evaluation on strictly disjoint data. A harmonized error analysis separating genuine errors from valid stylistic variation (tense, word order, Swiss orthography) yields a content WER (cWER) of 13.8%, counting only actual recognition failures. Bias-corrected estimation reduces this to 8.5%, suggesting the true error rate is roughly one third of measured WER. We demonstrate that published state-of-the-art Swiss German ASR results (17.1-17.5% WER) are inflated by benchmark contamination: a vanilla Whisper model self-trained on the ASGDTS test set with zero Swiss German data achieves 13.88% WER, surpassing all published systems. Experiments with Phi-4-multimodal show an even stronger memorization effect (3.9% WER), revealing that the benchmark primarily measures convention matching rather than dialectal comprehension. We release two models, a LoRA adapter (25.32% WER, 13.9% cWER) and a full fine-tuned model (25.60% WER, 13.8% cWER), among the few publicly available, honestly evaluated Whisper models for Swiss German, under Apache 2.0 with full reproducibility, requiring no institutional data agreements.
翻译:我们系统研究了针对瑞士德语音频转写任务微调OpenAI Whisper large-v3模型的过程,使用1,367小时广播语音与标准德语字幕作为弱监督信号。通过NVIDIA DGX Spark(Grace Blackwell架构,128 GB统一内存,最高1 PFLOP FP4性能)上开展的16轮迭代训练,我们对比了LoRA与全参数微调(1.55B参数模型)的效果,探究了幻觉现象的根本原因,并量化了数据质量、字幕对齐度和训练策略的影响。在严格无重叠数据上的诚实评估中,我们的最佳模型在"全瑞士德语方言测试集"(ASGDTS)上测得25.6%的词错误率(WER)。通过区分真实错误与有效文体变异(时态、语序、瑞士正字法)的协调误差分析,获得13.8%的内容词错误率(cWER),仅统计实际识别失败案例。偏差校正估计进一步将该值降至8.5%,表明真实错误率约为实测WER的三分之一。我们证明,已发表的瑞士德语ASR最优结果(17.1-17.5% WER)受到基准数据污染的影响:一个零瑞士德语训练数据的原生Whisper模型在ASGDTS测试集上自训练后达到13.88% WER,超越所有已发表系统。Phi-4-multimodal实验显示更强的记忆效应(3.9% WER),揭示该基准主要评估规范匹配能力而非方言理解能力。我们基于Apache 2.0协议发布两个模型——LoRA适配器(25.32% WER,13.9% cWER)和全参数微调模型(25.60% WER,13.8% cWER),这是少数公开可用且经过诚实评估的瑞士德语Whisper模型,具有完全可复现性且无需机构数据协议。