The rapid expansion of AI deployments has put organizational leaders in a decision maker's dilemma: they must govern these technologies without systematic evidence of how systems behave in their own environments. Predominant evaluation methods generate scalable, abstract measures of model capabilities but smooth over the heterogeneity of real world use, while user focused testing reveals rich contextual detail yet remains small in scale and loosely coupled to the mechanisms that shape model behavior. The Forum for Real World AI Measurement and Evaluation (FRAME) addresses this gap by combining large scale trials of AI systems with structured observation of how they are used in context, the outcomes they generate, and how those outcomes arise. By tracing the path from an AI system's output through its practical use and downstream effects, FRAME turns the heterogeneity of AI in use into a measurable signal rather than a trade off for achieving scale. FRAME establishes two core assets to accomplish this: a Testing Sandbox that captures AI use under real workflows at scale and a Metrics Hub that translates those traces into actionable indicators.

翻译：人工智能部署的快速扩张使组织领导者陷入决策者困境：他们必须在缺乏系统证据的情况下治理这些技术，无法确知系统在自身环境中的实际行为。主流评估方法虽能生成可扩展的模型能力抽象度量，却掩盖了现实应用场景的异质性；而以用户为中心的测试虽能揭示丰富的上下文细节，却规模有限且与塑造模型行为的机制耦合松散。真实世界人工智能测量与评估论坛（FRAME）通过结合大规模人工智能系统试验与结构化观察——涵盖其情境化使用方式、产生的实际结果及其形成机制——来弥合这一鸿沟。通过追踪人工智能系统输出经实际应用至下游影响的完整路径，FRAME将使用中人工智能的异质性转化为可测量信号，而非实现规模化的妥协代价。为此，FRAME构建了两大核心资产：可大规模捕捉真实工作流中人工智能使用状态的测试沙盒，以及将这些轨迹转化为可操作指标的度量中心。