Assessing the validity of a real-world system with respect to given quality criteria is a common yet costly task in industrial applications due to the vast number of required real-world tests. Validating such systems by means of simulation offers a promising and less expensive alternative, but requires an assessment of the simulation accuracy and therefore end-to-end measurements. Additionally, covariate shifts between simulations and actual usage can cause difficulties for estimating the reliability of such systems. In this work, we present a validation method that propagates bounds on distributional discrepancy measures through a composite system, thereby allowing us to derive an upper bound on the failure probability of the real system from potentially inaccurate simulations. Each propagation step entails an optimization problem, where -- for measures such as maximum mean discrepancy (MMD) -- we develop tight convex relaxations based on semidefinite programs. We demonstrate that our propagation method yields valid and useful bounds for composite systems exhibiting a variety of realistic effects. In particular, we show that the proposed method can successfully account for data shifts within the experimental design as well as model inaccuracies within the simulation.

翻译：评估真实系统在给定质量标准下的有效性是工业应用中一项常见但成本高昂的任务，原因在于所需的大量真实环境测试。通过仿真验证此类系统提供了一种有前景且成本较低的替代方案，但需要评估仿真精度，因此必须进行端到端测量。此外，仿真与实际使用之间的协变量偏移可能会给估计此类系统的可靠性带来困难。在本工作中，我们提出了一种验证方法，该方法通过复合系统传播分布差异测度的边界，从而允许我们从可能不准确的仿真中推导出真实系统失效率的上界。每个传播步骤涉及一个优化问题——针对最大均值差异（MMD）等测度，我们基于半定规划开发了紧致的凸松弛方法。我们证明，对于呈现多种现实效应的复合系统，该传播方法能生成有效且有用的边界。特别地，研究表明所提方法能够成功解释实验设计中的数据偏移以及仿真中的模型不准确性。