The advent of advanced AI underscores the urgent need for comprehensive safety evaluations, necessitating collaboration across communities (i.e., AI, software engineering, and governance). However, divergent practices and terminologies across these communities, combined with the complexity of AI systems-of which models are only a part-and environmental affordances (e.g., access to tools), obstruct effective communication and comprehensive evaluation. This paper proposes a framework for AI system evaluation comprising three components: 1) harmonised terminology to facilitate communication across communities involved in AI safety evaluation; 2) a taxonomy identifying essential elements for AI system evaluation; 3) a mapping between AI lifecycle, stakeholders, and requisite evaluations for accountable AI supply chain. This framework catalyses a deeper discourse on AI system evaluation beyond model-centric approaches.

翻译：先进人工智能的出现凸显了进行全面安全评估的紧迫性，这需要人工智能、软件工程与治理等跨领域协作。然而，各领域间迥异的实践方法与术语体系，加之AI系统（其中模型仅为组成部分）本身的复杂性及环境赋能（如工具调用权限），阻碍了有效沟通与全面评估。本文提出由三部分构成的AI系统评估框架：1）统一术语体系以促进AI安全评估所涉领域的沟通；2）定义AI系统评估核心要素的分类法；3）建立AI生命周期、利益相关方与负责任AI供应链必要评估间的映射关系。该框架旨在推动超越以模型为中心的范式，深化对AI系统评估的探讨。