Create an idea, prototype it, evaluate if users like it, then learn. It is the circle of business. If AI can operate in all parts of the circle, it will enable rapid iteration and learning speeds for businesses. Experiment platforms that deploy experiments to evaluate return on investment for businesses are abundant, but systems that help businesses learn personalization, mechanisms, and what to ideate next, are rare. Among technologies that do exist, they cannot be well orchestrated in a single software interface that can be safely and efficiently leveraged by an AI agent. These challenges make it difficult to teach an AI agent how to learn within a robust experimentation framework, and difficult for an AI agent to operate and iterate for the business. We offer a two part solution: one half that is rooted in mathematical reductions to contain complexity, and one half that is rooted in software design to optimize for orchestration, software safety, and multiplicity. Our solution, a software framework, moves beyond the simple treatment effect computed as a difference in means. To create a better understanding of a business and its customers, we enrich causal analysis with heterogeneous effects, policy algorithms, mediation analysis, and forecasts of effects. To have an AI complete the iteration cycle faster, we further enrich the analysis with variance reduction and anytime valid inference. The enrichments are made compatible across different types of experiments, and are presented in a single software interface that is usable in an AI agent. We evaluate the approach on various objectives in experiment analysis, and show that the framework improves code correctness, reduces lines of code, and is more performant than a baseline analysis constructed by a vanilla agent.

翻译：构思一个想法，构建原型，评估用户是否喜欢，然后学习——这是商业的循环。若人工智能能够参与该循环的所有环节，企业便能够实现快速的迭代与学习。当前，部署实验以评估商业投资回报率的实验平台比比皆是，但能够帮助企业学习个性化策略、机制设计以及下一步创新方向的系统却十分罕见。即便存在相关技术，它们也难以被整合到一个统一、安全且能被人工智能智能体高效利用的软件接口中。这些挑战使得在稳健的实验框架内训练人工智能智能体学习变得困难，也使其难以为企业进行自动化操作与迭代。我们提出一个双模块解决方案：其一基于数学简化以控制复杂度，其二基于软件设计以优化编排、安全性与兼容性。我们的解决方案是一个软件框架，其超越了基于均值差异计算的简单处理效应。为更深入地理解企业及其客户，我们通过异质性效应、策略算法、中介分析及效应预测来丰富因果分析。为让人工智能更快完成迭代周期，我们进一步引入方差缩减与随时有效推断来增强分析。这些增强功能兼容不同类型的实验，并集成在统一的软件接口中，可供人工智能智能体直接使用。我们在实验分析的多项目标上评估了该方法，结果表明该框架提升了代码正确性，减少了代码行数，且性能优于由基础智能体构建的基线分析。