TestGen automatically generates unit tests, carved from serialized observations of complex objects, observed during app execution. We describe the development and deployment of TestGen at Meta. In particular, we focus on the scalability challenges overcome during development in order to deploy observation-based test carving at scale in industry. So far, TestGen has landed 518 tests into production, which have been executed 9,617,349 times in continuous integration, finding 5,702 faults. Meta is currently in the process of more widespread deployment. Our evaluation reveals that, when carving its observations from 4,361 reliable end-to-end tests, TestGen was able to generate tests for at least 86\% of the classes covered by end-to-end tests. Testing on 16 Kotlin Instagram app-launch-blocking tasks demonstrated that the TestGen tests would have trapped 13 of these before they became launch blocking.
翻译:TestGen可自动生成单元测试,这些测试源于对应用执行过程中观察到的复杂对象序列化观测数据的提取。本文描述了TestGen在Meta的开发与部署过程,重点阐述了为实现在工业界大规模部署基于观察的测试提取技术所克服的可扩展性挑战。截至目前,TestGen已向生产环境提交518个测试用例,这些测试在持续集成中累计执行9,617,349次,发现5,702个缺陷。Meta目前正推进该工具的更大规模部署。评估表明,当从4,361个可靠的端到端测试中提取观测数据时,TestGen能够为至少86%的端到端测试覆盖的类生成测试用例。针对16个Kotlin Instagram应用启动阻塞任务的测试显示,TestGen生成的测试可捕获其中13个任务在演变为启动阻塞问题前的潜在缺陷。