How are AI assistants being used in the real world? While model providers in theory have a window into this impact via their users' data, both privacy concerns and practical challenges have made analyzing this data difficult. To address these issues, we present Clio (Claude insights and observations), a privacy-preserving platform that uses AI assistants themselves to analyze and surface aggregated usage patterns across millions of conversations, without the need for human reviewers to read raw conversations. We validate this can be done with a high degree of accuracy and privacy by conducting extensive evaluations. We demonstrate Clio's usefulness in two broad ways. First, we share insights about how models are being used in the real world from one million Claude.ai Free and Pro conversations, ranging from providing advice on hairstyles to providing guidance on Git operations and concepts. We also identify the most common high-level use cases on Claude.ai (coding, writing, and research tasks) as well as patterns that differ across languages (e.g., conversations in Japanese discuss elder care and aging populations at higher-than-typical rates). Second, we use Clio to make our systems safer by identifying coordinated attempts to abuse our systems, monitoring for unknown unknowns during critical periods like launches of new capabilities or major world events, and improving our existing monitoring systems. We also discuss the limitations of our approach, as well as risks and ethical concerns. By enabling analysis of real-world AI usage, Clio provides a scalable platform for empirically grounded AI safety and governance.

翻译：AI助手在现实世界中是如何被使用的？尽管模型提供商理论上可以通过用户数据了解其影响，但隐私问题和实际挑战使得分析这些数据变得困难。为解决这些问题，我们提出了Clio（Claude洞察与观测平台）——一个隐私保护平台，该平台利用AI助手自身来分析和呈现数百万次对话中的聚合使用模式，无需人工审核员阅读原始对话内容。通过大量评估验证，我们证明该方法可在高度准确性和隐私保护的前提下实现。我们从两个宏观维度展示Clio的实用价值：首先，基于一百万次Claude.ai免费版与专业版对话，我们揭示了模型在现实场景中的使用情况——从发型建议到Git操作与概念指导。我们识别出Claude.ai上最高频的宏观使用场景（编程、写作与研究任务），以及跨语言差异模式（例如日语对话中讨论老年护理与人口老龄化的频率显著高于常规水平）。其次，我们运用Clio提升系统安全性：识别协同滥用系统的企图，在关键时期（如新功能发布或重大世界事件期间）监测未知风险，并改进现有监控体系。同时，我们探讨了该方法的局限性、潜在风险与伦理问题。通过实现对现实世界AI使用的分析，Clio为基于实证的AI安全与治理提供了可扩展的平台。