AI-assisted coding tools have altered software production. At Meta, significant lines of code per human-landed diff grew by 105.9% year over year and per-developer diff volume rose 51%, with agentic AI responsible for over 80% of that growth. Meanwhile, the share of diffs receiving timely review has declined, exposing a widening gap between code supply and reviewer bandwidth. We ask three questions that progress from feasibility through calibration to impact: (1) can risk-stratified automation operate at scale across diverse organizations, (2) how does tuning the risk threshold affect the trade-off between automation yield and safety, and (3) to what extent does automated review reduce end-to-end latency for AI-generated changes? We deployed RADAR (Risk Aware Diff Auto Review), a multi-stage funnel that classifies each diff by authorship and source type, applies eligibility gates, static heuristics, a machine-learned Diff Risk Score, LLM-based Automated Code Review, and deterministic validation before landing qualifying changes. We evaluate RADAR through telemetry covering 535K+ RADAR-reviewed diffs, observational before-after comparisons for policy changes, and difference-in-differences analysis of efficiency outcomes. RADAR has reviewed 535K+ diffs and landed 331K+. Relaxing the Diff Risk Score threshold from the 25th to the 50th percentile increased the approve rate to 60.31%. The revert rate for RADAR-reviewed diffs is 1/3 that of non-RADAR diffs, and the Production Incident rate is 1/50 that of non-RADAR diffs. RADAR reduces median time to close by over 330% and median diff review wall time by 35%. Risk-aware layered automation can materially reduce review bottlenecks created by AI-driven code growth without compromising production safety.
翻译:AI辅助编程工具已深刻改变了软件生产方式。在Meta,由人类提交的代码差异(diff)中,每项差异的平均代码行数同比增长105.9%,每位开发者的差异提交量增长51%,其中智能代理AI贡献了超过80%的增长。与此同时,获得及时审查的差异比例却在下降,暴露出代码供应与审查人员带宽之间日益扩大的差距。我们提出了三个递进式研究问题——从可行性到校准再到影响评估:(1)跨不同组织的大规模风险分层自动化是否可行;(2)调整风险阈值如何影响自动化产出与安全性之间的权衡;(3)自动化审查能在多大程度上降低AI生成变更的端到端延迟?我们部署了RADAR(风险感知差异自动审查)系统,这是一个多阶段漏斗结构:首先按作者身份和来源类型对差异进行分类,随后依次通过资格门控、静态启发式规则、机器学习差异风险评分、基于大语言模型的自动化代码审查,以及在变更落地前进行确定性验证。我们通过覆盖超过53.5万次RADAR审查差异的遥测数据、政策变更前后的观察对比以及效率结果的倍差法分析,对RADAR进行了评估。RADAR已审查超过53.5万次差异,其中33.1万余次获得批准落地。将差异风险评分阈值从第25百分位放宽至第50百分位后,批准率提升至60.31%。经RADAR审查的差异的回滚率仅为非RADAR差异的三分之一,生产事故率仅为非RADAR差异的五十分之一。RADAR将差异中位关闭时间缩短超过330%,中位审查周期时间缩短35%。采用风险感知的分层自动化策略,能够在不牺牲生产安全的前提下,有效缓解由AI驱动的代码增长所导致的审查瓶颈。