The rapid expansion of AI deployments has put organizational leaders in a decision maker's dilemma: they must govern these technologies without systematic evidence of how systems behave in their own environments. Predominant evaluation methods generate scalable, abstract measures of model capabilities but smooth over the heterogeneity of real world use, while user focused testing reveals rich contextual detail yet remains small in scale and loosely coupled to the mechanisms that shape model behavior. The Forum for Real World AI Measurement and Evaluation (FRAME) addresses this gap by combining large scale trials of AI systems with structured observation of how they are used in context, the outcomes they generate, and how those outcomes arise. By tracing the path from an AI system's output through its practical use and downstream effects, FRAME turns the heterogeneity of AI in use into a measurable signal rather than a trade off for achieving scale. FRAME establishes two core assets to accomplish this: a Testing Sandbox that captures AI use under real workflows at scale and a Metrics Hub that translates those traces into actionable indicators.

翻译：人工智能部署的迅速扩展使组织领导者陷入决策者困境：他们必须在缺乏系统证据的情况下治理这些技术——这些证据本应揭示系统在其自身环境中的实际行为表现。主流评估方法虽然能生成可扩展的抽象模型能力度量，却忽略了真实世界应用场景的异质性；而聚焦用户的测试虽能呈现丰富的语境细节，但其规模有限且与塑造模型行为的内在机制耦合松散。真实世界人工智能测量与评估论坛（FRAME）通过将大规模AI系统试验与结构化观察相结合来弥合这一鸿沟——这些观察涵盖系统在特定语境中的使用、产生的实际结果以及这些结果的成因。通过追踪从AI系统输出到实际应用及下游影响的完整路径，FRAME将使用中AI的异质性转化为可测量的信号，而非为了实现规模化而付出的代价。FRAME为此建立了两项核心资产：能够在大规模真实工作流程下捕获AI使用的测试沙盒，以及将轨迹转化为可操作指标的度量中心。