General-purpose AI evaluations have been proposed as a promising way of identifying and mitigating systemic risks posed by AI development and deployment. While GPAI evaluations play an increasingly central role in institutional decision- and policy-making -- including by way of the European Union AI Act's mandate to conduct evaluations on GPAI models presenting systemic risk -- no standards exist to date to promote their quality or legitimacy. To strengthen GPAI evaluations in the EU, which currently constitutes the first and only jurisdiction that mandates GPAI evaluations, we outline four desiderata for GPAI evaluations: internal validity, external validity, reproducibility, and portability. To uphold these desiderata in a dynamic environment of continuously evolving risks, we propose a dedicated EU GPAI Evaluation Standards Taskforce, to be housed within the bodies established by the EU AI Act. We outline the responsibilities of the Taskforce, specify the GPAI provider commitments that would facilitate Taskforce success, discuss the potential impact of the Taskforce on global AI governance, and address potential sources of failure that policymakers should heed.

翻译：通用人工智能评估已被提出作为一种有前景的方法，用以识别和缓解人工智能开发与部署所带来的系统性风险。尽管通用人工智能评估在机构决策和政策制定中扮演着日益核心的角色——包括通过欧盟《人工智能法案》对呈现系统性风险的通用人工智能模型强制进行评估——但迄今为止，尚无任何标准来提升其质量或合法性。为了加强欧盟的通用人工智能评估（欧盟是目前第一个也是唯一一个强制要求进行通用人工智能评估的司法管辖区），我们概述了通用人工智能评估的四个理想特性：内部效度、外部效度、可复现性和可移植性。为了在不断演化的风险动态环境中维护这些理想特性，我们提议在欧盟《人工智能法案》设立的机构内成立一个专门的欧盟通用人工智能评估标准工作组。我们概述了该工作组的职责，明确了有助于工作组取得成功的通用人工智能提供方承诺，讨论了工作组对全球人工智能治理的潜在影响，并指出了政策制定者应注意的潜在失败根源。