AI evaluation results are produced at scale but reported inconsistently across leaderboards, model cards, benchmark papers, and company blogs. The cost is interpretive: readers cannot reliably compare results across sources, identify what a report omits, or trace an aggregate claim to its underlying evidence. Recent efforts address isolated components but leave three gaps: they cover only narrow slices of the evaluation lifecycle and do not compose into a single interpretable record; they specify static representations that do not differentiate the questions different stakeholders bring to the same evidence; and they remain proposals on paper, lacking the extraction infrastructure required for adoption at scale. We present \EvalCards{}, an operational reporting layer that composes benchmark metadata, evaluation run data, and model metadata into a unified record. We (1) derive a reporting schema from a structured review of 52 papers and 10 stakeholder interviews, (2) implement four interpretive signals (reproducibility, documentation completeness, provenance and risk, and score comparability), rendered through reader modes calibrated to research and non-research audiences, and (3) deploy a monitoring tool that applies \EvalCards{} across 5,816 models, 635 benchmarks, and 101,843 results, surfacing systematic gaps in current reporting practice.
翻译:AI评估结果大规模产出,但在排行榜、模型卡、基准测试论文和公司博客中的报告方式却不一致。其代价是解释性的:读者无法可靠地跨来源比较结果、识别报告遗漏的内容,或将聚合性论断追溯至其底层证据。近年来的努力解决了孤立组件问题,但仍存在三个空白:它们仅涵盖评估生命周期的狭窄片段,无法组合成单一可解释的记录;它们指定了静态表征形式,无法区分不同利益相关者对同一证据所提出的问题;它们仍停留在纸面提议上,缺乏规模化采用所需的抽取基础设施。我们提出EvalCards(一种运作性报告层),将基准元数据、评估运行数据和模型元数据组合成统一记录。我们:(1) 基于对52篇论文和10位利益相关者访谈的结构化评审推导出报告模式;(2) 实现四种解释性信号(可重复性、文档完备性、溯源与风险、分数可比性),通过针对研究与非研究受众校准的阅读模式进行呈现;(3) 部署一个监控工具,在5,816个模型、635个基准和101,843个结果上应用EvalCards,揭示当前报告实践中的系统性空白。