Leaderboard scores on public benchmarks have been steadily rising and converging, with many frontier language models now separated by only marginal differences. However, these scores often fail to match users' day to day experience, because system prompts, output protocols, and interaction modes evolve under routine iteration, and in agentic multi step pipelines small protocol shifts can trigger disproportionate failures, leaving practitioners uncertain about which model to deploy. We propose CreditAudit, a deployment oriented credit audit framework that evaluates models under a family of semantically aligned and non adversarial system prompt templates across multiple benchmarks, reporting mean ability as average performance across scenarios and scenario induced fluctuation sigma as a stability risk signal, and further mapping volatility into interpretable credit grades from AAA to BBB via cross model quantiles with diagnostics that mitigate template difficulty drift. Controlled experiments on GPQA, TruthfulQA, and MMLU Pro show that models with similar mean ability can exhibit substantially different fluctuation, and stability risk can overturn prioritization decisions in agentic or high failure cost regimes. By providing a 2D and grade based language for regime specific selection, CreditAudit supports tiered deployment and more disciplined allocation of testing and monitoring effort, enabling more objective and trustworthy model evaluation for real world use.

翻译：公开基准测试的排行榜分数持续上升并趋于收敛，当前许多前沿语言模型的得分差异已微乎其微。然而，这些分数常与用户的日常体验不符，原因在于系统提示、输出协议及交互模式在常规迭代中持续演进，且在智能体多步流程中，细微的协议偏移可能引发不成比例的故障，导致实践者难以确定应部署何种模型。我们提出CreditAudit——一个面向部署的信用审计框架，该框架通过一系列语义对齐且非对抗性的系统提示模板在多个基准上评估模型：报告平均能力作为跨场景的性能均值，并以场景诱导的波动标准差作为稳定性风险信号；进而通过跨模型分位数将波动性映射为从AAA至BBB的可解释信用等级，其诊断机制可缓解模板难度漂移。在GPQA、TruthfulQA和MMLU Pro上的受控实验表明，具有相近平均能力的模型可能表现出显著不同的波动性，且在智能体或高故障代价场景中，稳定性风险可能颠覆原有的优先级决策。通过提供基于二维指标与信用等级的领域专用选择语言，CreditAudit支持分层部署及更规范的测试与监控资源分配，从而为实际应用场景实现更客观、可信的模型评估。