Automatic speech recognition (ASR) systems have achieved near-human accuracy on curated benchmarks, yet still fail in real-world voice agents under conditions that current evaluations do not systematically cover. Without diagnostic tools that isolate specific failure factors, practitioners cannot anticipate which conditions, in which languages, will cause what degree of degradation. We introduce WildASR, a multilingual (four-language) diagnostic benchmark sourced entirely from real human speech that factorizes ASR robustness along three axes: environmental degradation, demographic shift, and linguistic diversity. Evaluating seven widely used ASR systems, we find severe and uneven performance degradation, and model robustness does not transfer across languages or conditions. Critically, models often hallucinate plausible but unspoken content under partial or degraded inputs, creating concrete safety risks for downstream agent behavior. Our results demonstrate that targeted, factor-isolated evaluation is essential for understanding and improving ASR reliability in production systems. Besides the benchmark itself, we also present three analytical tools that practitioners can use to guide deployment decisions.

翻译：自动语音识别（ASR）系统在精心构建的基准测试中已接近人类水平，但在当前评估体系未系统覆盖的真实语音代理场景中仍会失效。由于缺乏隔离特定失败因素的诊断工具，从业者无法预判何种语言在何种条件下会导致何种程度的性能退化。我们提出WildASR——一个完全源于真实人类语音的多语言（四种语言）诊断基准，沿环境退化、人口统计差异和语言多样性三个维度分解ASR鲁棒性。通过评估七个广泛使用的ASR系统，我们发现严重且不均衡的性能退化，且模型鲁棒性在不同语言或条件下不具备迁移性。尤为关键的是，在部分输入或退化输入条件下，模型常会幻觉性生成合理但未实际存在的内容，为下游智能代理行为带来具体安全风险。我们的研究表明，针对性的因子隔离评估对于理解与改进生产系统中ASR可靠性至关重要。除基准本身外，我们还提供三种分析工具，从业者可借以指导部署决策。