Amortized Bayesian inference trains neural networks to solve stochastic inference problems using model simulations, thereby making it possible to rapidly perform Bayesian inference for any newly observed data. However, current simulation-based amortized inference methods are simulation-hungry and inflexible: They require the specification of a fixed parametric prior, simulator, and inference tasks ahead of time. Here, we present a new amortized inference method -- the Simformer -- which overcomes these limitations. By training a probabilistic diffusion model with transformer architectures, the Simformer outperforms current state-of-the-art amortized inference approaches on benchmark tasks and is substantially more flexible: It can be applied to models with function-valued parameters, it can handle inference scenarios with missing or unstructured data, and it can sample arbitrary conditionals of the joint distribution of parameters and data, including both posterior and likelihood. We showcase the performance and flexibility of the Simformer on simulators from ecology, epidemiology, and neuroscience, and demonstrate that it opens up new possibilities and application domains for amortized Bayesian inference on simulation-based models.

翻译：摊销贝叶斯推断通过模型仿真训练神经网络以解决随机推断问题，从而能够对任何新观测数据快速执行贝叶斯推断。然而，当前基于仿真的摊销推断方法存在仿真依赖性强与灵活性不足的缺陷：它们需要预先确定固定的参数化先验分布、仿真器及推断任务。本文提出一种新型摊销推断方法——Simformer，该方法成功克服了这些局限性。通过采用Transformer架构训练概率扩散模型，Simformer在基准任务上超越了当前最先进的摊销推断方法，并展现出显著更强的灵活性：可应用于函数值参数模型，能够处理含缺失数据或非结构化数据的推断场景，且能对参数与数据联合分布的任意条件分布（包括后验分布与似然函数）进行采样。我们在生态学、流行病学和神经科学的仿真器上验证了Simformer的性能与灵活性，证明该方法为基于仿真模型的摊销贝叶斯推断开辟了新的可能性与应用领域。