The optimization of a black-box simulator over control parameters $\mathbf{x}$ arises in a myriad of scientific applications. In such applications, the simulator often takes the form $f(\mathbf{x},\boldsymbol{\theta})$, where $\boldsymbol{\theta}$ are parameters that are uncertain in practice. Robust optimization aims to optimize the objective $\mathbb{E}[f(\mathbf{x},\boldsymbol{\Theta})]$, where $\boldsymbol{\Theta} \sim \mathcal{P}$ is a random variable that models uncertainty on $\boldsymbol{\theta}$. For this, existing black-box methods typically employ a two-stage approach for selecting the next point $(\mathbf{x},\boldsymbol{\theta})$, where $\mathbf{x}$ and $\boldsymbol{\theta}$ are optimized separately via different acquisition functions. As such, these approaches do not employ a joint acquisition over $(\mathbf{x},\boldsymbol{\theta})$, and thus may fail to fully exploit control-to-noise interactions for effective robust optimization. To address this, we propose a new Bayesian optimization method called Targeted Variance Reduction (TVR). The TVR leverages a novel joint acquisition function over $(\mathbf{x},\boldsymbol{\theta})$, which targets variance reduction on the objective within the desired region of improvement. Under a Gaussian process surrogate on $f$, the TVR acquisition can be evaluated in closed form, and reveals an insightful exploration-exploitation-precision trade-off for robust black-box optimization. The TVR can further accommodate a broad class of non-Gaussian distributions on $\mathcal{P}$ via a careful integration of normalizing flows. We demonstrate the improved performance of TVR over the state-of-the-art in a suite of numerical experiments and an application to the robust design of automobile brake discs under operational uncertainty.

翻译：黑箱模拟器在控制参数 $\mathbf{x}$ 上的优化出现在众多科学应用中。此类应用中，模拟器通常采用形式 $f(\mathbf{x},\boldsymbol{\theta})$，其中 $\boldsymbol{\theta}$ 是实际中存在不确定性的参数。鲁棒优化旨在优化目标 $\mathbb{E}[f(\mathbf{x},\boldsymbol{\Theta})]$，其中 $\boldsymbol{\Theta} \sim \mathcal{P}$ 是对 $\boldsymbol{\theta}$ 不确定性建模的随机变量。为此，现有黑箱方法通常采用两阶段策略选择下一个观测点 $(\mathbf{x},\boldsymbol{\theta})$，即通过不同采集函数分别优化 $\mathbf{x}$ 和 $\boldsymbol{\theta}$。这类方法未对 $(\mathbf{x},\boldsymbol{\theta})$ 进行联合采集，因此可能无法充分利用控制与噪声的交互作用以实现高效鲁棒优化。针对该问题，我们提出一种新的贝叶斯优化方法——目标方差缩减（Targeted Variance Reduction, TVR）。TVR 利用针对 $(\mathbf{x},\boldsymbol{\theta})$ 的新型联合采集函数，该函数旨在减少期望改善区域内目标函数的方差。在基于高斯过程代理模型建模 $f$ 的条件下，TVR 采集函数可解析求解，并揭示了鲁棒黑箱优化中探索-利用-精度的权衡关系。通过精心集成归一化流，TVR 还可处理 $\mathcal{P}$ 上广泛的非高斯分布。我们通过一系列数值实验及考虑运行不确定性的汽车刹车盘鲁棒设计应用，展示了 TVR 相比现有最优方法的性能提升。