AI evaluation results are produced at scale but reported inconsistently across leaderboards, model cards, benchmark papers, and company blogs. The cost is interpretive: readers cannot reliably compare results across sources, identify what a report omits, or trace an aggregate claim to its underlying evidence. Recent efforts address isolated components but leave three gaps: they cover only narrow slices of the evaluation lifecycle and do not compose into a single interpretable record; they specify static representations that do not differentiate the questions different stakeholders bring to the same evidence; and they remain proposals on paper, lacking the extraction infrastructure required for adoption at scale. We present \EvalCards{}, an operational reporting layer that composes benchmark metadata, evaluation run data, and model metadata into a unified record. We (1) derive a reporting schema from a structured review of 52 papers and 10 stakeholder interviews, (2) implement four interpretive signals (reproducibility, documentation completeness, provenance and risk, and score comparability), rendered through reader modes calibrated to research and non-research audiences, and (3) deploy a monitoring tool that applies \EvalCards{} across 5,816 models, 635 benchmarks, and 101,843 results, surfacing systematic gaps in current reporting practice.
翻译:AI评估结果虽大规模生成,但在排行榜、模型卡片、基准论文及公司博客中的报告方式却存在不一致性。其代价在于解释性:读者无法可靠地跨来源比较结果、识别报告中的遗漏之处,或将总体结论追溯至其底层证据。近期工作虽针对孤立组件进行了改进,但仍存在三大缺口:仅覆盖评估生命周期中的狭窄片段,且无法组合成单一可解释记录;采用静态表征方式,未能区分不同利益相关者对同一证据所提出的差异化问题;并停留在纸上提案阶段,缺乏规模化应用所需的提取基础设施。我们提出评估卡片(EvalCards),这是一种可操作的报告层,能将基准元数据、评估运行数据和模型元数据组合成统一记录。我们:(1) 基于对52篇论文的结构化综述及10位利益相关者访谈,推导出报告模式;(2) 实现四种解释性信号(可复现性、文档完整性、溯源与风险、以及分数可比性),并通过为研究与大众受众校准的阅读模式进行呈现;(3) 部署监测工具,在5,816个模型、635个基准和101,843项结果上应用评估卡片,揭示当前报告实践中的系统性缺失。