Following the AI Seoul Summit in 2024, twelve AI companies published frontier AI safety frameworks (Frameworks) outlining their approaches to managing catastrophic risks from advanced AI systems. Emerging legislation increasingly treats these Frameworks as external accountability mechanisms, incorporating them into reporting requirements. But what do the Frameworks actually commit each company to do? This study assesses 12 Frameworks, using 65 weighted criteria, across four dimensions: risk identification, risk analysis \& evaluation, risk treatment, and risk governance. Our criteria adapt established risk management principles from other high-risk industries (e.g. aviation, nuclear power) to the frontier AI context, following Campos et al. (2025). Overall scores range from 34% (Anthropic) to 8% (Cohere), with a median of 18%. Many aspects are missing or under-specified. These low scores may be natural given the nascency of AI risk management compared to industries with decades of practice. Nonetheless, current Frameworks are limited as accountability functions, with vague commitments that make it difficult to predict company decisions, assess whether planned responses are adequate, or determine whether commitments have been kept. Still, higher scores appear feasible within current constraints: a company adopting all leading practices currently adopted across their peers would score 54%, which is triple the current median.

翻译：继2024年首尔人工智能峰会之后，十二家AI公司发布了前沿AI安全框架，概述了它们管理先进AI系统带来的灾难性风险的方法。新兴立法越来越多地将这些框架视为外部问责机制，并将其纳入报告要求中。但这些框架实际要求每家公司做出哪些承诺？本研究评估了12个框架，使用65项加权标准，涵盖四个维度：风险识别、风险分析与评估、风险处理和风险治理。我们的标准借鉴了其他高风险行业（如航空、核电）已建立的风险管理原则，并针对前沿AI背景进行了调整，遵循Campos等人（2025）的研究。总体得分从34%（Anthropic）到8%（Cohere）不等，中位数为18%。许多方面缺失或规定不足。鉴于AI风险管理相比拥有数十年实践的行业仍处于起步阶段，这些低分可能是自然的。尽管如此，当前框架作为问责功能有限，模糊的承诺使得难以预测公司决策、评估计划应对措施是否充分，或确定承诺是否得到履行。然而，在当前约束下实现更高分数似乎可行：一家采用同行公司所有领先实践的公司将获得54%的分数，是当前中位数的三倍。