Before autonomous systems can be deployed in safety-critical applications, we must be able to understand and verify the safety of these systems. For cases where the risk or cost of real-world testing is prohibitive, we propose a simulation-based framework for a) predicting ways in which an autonomous system is likely to fail and b) automatically adjusting the system's design to preemptively mitigate those failures. We frame this problem through the lens of approximate Bayesian inference and use differentiable simulation for efficient failure case prediction and repair. We apply our approach on a range of robotics and control problems, including optimizing search patterns for robot swarms and reducing the severity of outages in power transmission networks. Compared to optimization-based falsification techniques, our method predicts a more diverse, representative set of failure modes, and we also find that our use of differentiable simulation yields solutions that have up to 10x lower cost and requires up to 2x fewer iterations to converge relative to gradient-free techniques. Code and videos can be found at https://mit-realm.github.io/breaking-things/

翻译：在将自主系统部署于安全关键应用之前，我们必须能够理解并验证这些系统的安全性。针对现实世界测试风险或成本过高的情况，我们提出一种基于仿真的框架，用于：a) 预测自主系统可能失效的方式，b) 自动调整系统设计以预先缓解这些故障。我们将此问题置于近似贝叶斯推断的视角下，利用可微仿真实现高效的失效案例预测与修复。我们将该方法应用于一系列机器人学与控制问题，包括优化机器人集群的搜索模式以及降低输电网故障严重程度。与基于优化的伪造技术相比，我们的方法能预测更具多样性和代表性的失效模式集；同时发现，相较于无梯度方法，使用可微仿真获得的解决方案成本可降低达10倍，且收敛所需迭代次数减少达2倍。代码与视频访问地址：https://mit-realm.github.io/breaking-things/