Bayesian inference usually requires running potentially costly inference procedures separately for every new observation. In contrast, the idea of amortized Bayesian inference is to initially invest computational cost in training an inference network on simulated data, which can subsequently be used to rapidly perform inference (i.e., to return estimates of posterior distributions) for new observations. This approach has been applied to many real-world models in the sciences and engineering, but it is unclear how robust the approach is to adversarial perturbations in the observed data. Here, we study the adversarial robustness of amortized Bayesian inference, focusing on simulation-based estimation of multi-dimensional posterior distributions. We show that almost unrecognizable, targeted perturbations of the observations can lead to drastic changes in the predicted posterior and highly unrealistic posterior predictive samples, across several benchmark tasks and a real-world example from neuroscience. We propose a computationally efficient regularization scheme based on penalizing the Fisher information of the conditional density estimator, and show how it improves the adversarial robustness of amortized Bayesian inference.

翻译：贝叶斯推理通常需要对每个新观测单独运行可能代价高昂的推理过程。相比之下，摊销贝叶斯推理的思想是：预先投入计算成本在模拟数据上训练推理网络，之后该网络可快速对新观测进行推理（即返回后验分布的估计值）。这一方法已应用于科学与工程中的许多实际模型，但其对观测数据中对抗性扰动的鲁棒性尚不明确。本文研究摊销贝叶斯推理的对抗鲁棒性，重点关注基于模拟的多维后验分布估计。我们证明，在多个基准任务及一个神经科学实际案例中，几乎不可察觉的针对性观测扰动会导致预测后验发生剧变，并产生极不现实的预测样本。我们提出了一种基于惩罚条件密度估计器费舍尔信息的计算高效正则化方案，并展示了它如何提升摊销贝叶斯推理的对抗鲁棒性。