Simulation-based inference (SBI) offers a flexible and general approach to performing Bayesian inference: In SBI, a neural network is trained on synthetic data simulated from a model and used to rapidly infer posterior distributions for observed data. A key goal for SBI is to achieve accurate inference with as few simulations as possible, especially for expensive simulators. In this work, we address this challenge by repurposing recent probabilistic foundation models for tabular data: We show how tabular foundation models -- specifically TabPFN -- can be used as pre-trained autoregressive conditional density estimators for SBI. We propose Neural Posterior Estimation with Prior-data Fitted Networks (NPE-PFN) and show that it is competitive with current SBI approaches in terms of accuracy for both benchmark tasks and two complex scientific inverse problems. Crucially, it often substantially outperforms them in terms of simulation efficiency, sometimes requiring orders of magnitude fewer simulations. NPE-PFN eliminates the need for inference network selection, training, and hyperparameter tuning. We also show that it exhibits superior robustness to model misspecification and can be scaled to simulation budgets that exceed the context size limit of TabPFN. NPE-PFN provides a new direction for SBI, where training-free, general-purpose inference models offer efficient, easy-to-use, and flexible solutions for a wide range of stochastic inverse problems.

翻译：基于仿真的推断（SBI）为执行贝叶斯推断提供了一种灵活通用的方法：在SBI中，神经网络通过模型生成的合成数据进行训练，并用于快速推断观测数据的后验分布。SBI的一个关键目标是以尽可能少的仿真次数实现精确推断，尤其针对计算成本高昂的仿真器。本研究通过改造近期提出的表格数据概率基础模型应对这一挑战：我们展示了表格基础模型——特别是TabPFN——如何作为预训练的自回归条件密度估计器应用于SBI。我们提出基于先验数据拟合网络的神经后验估计方法（NPE-PFN），并证明该方法在基准任务和两个复杂科学反问题中，其推断精度与当前主流SBI方法相当。更重要的是，该方法在仿真效率方面通常显著优于现有方法，有时可减少数个数量级的仿真需求。NPE-PFN无需进行推断网络选择、训练及超参数调优。研究还表明该方法对模型误设具有更强的鲁棒性，并能扩展至超越TabPFN上下文长度限制的仿真规模。NPE-PFN为SBI领域开辟了新方向，其中免训练、通用型的推断模型为广泛随机反问题提供了高效、易用且灵活的解决方案。