Foundation models based on prior-data fitted networks (PFNs) have shown strong empirical performance in causal inference by framing the task as an in-context learning problem.However, it is unclear whether PFN-based causal estimators provide uncertainty quantification that is consistent with classical frequentist estimators. In this work, we address this gap by analyzing the frequentist consistency of PFN-based estimators for the average treatment effect (ATE). (1) We show that existing PFNs, when interpreted as Bayesian ATE estimators, can exhibit prior-induced confounding bias: the prior is not asymptotically overwritten by data, which, in turn, prevents frequentist consistency. (2) As a remedy, we suggest employing a calibration procedure based on a one-step posterior correction (OSPC). We show that the OSPC helps to restore frequentist consistency and can yield a semi-parametric Bernstein-von Mises theorem for calibrated PFNs (i.e., both the calibrated PFN-based estimators and the classical semi-parametric efficient estimators converge in distribution with growing data size). (3) Finally, we implement OSPC through tailoring martingale posteriors on top of the PFNs. In this way, we are able to recover functional nuisance posteriors from PFNs, required by the OSPC. In multiple (semi-)synthetic experiments, PFNs calibrated with our martingale posterior OSPC produce ATE uncertainty that (i) asymptotically matches frequentist uncertainty and (ii) is well calibrated in finite samples in comparison to other Bayesian ATE estimators.

翻译：基于先验数据拟合网络（PFNs）的基础模型通过将因果推断任务构建为上下文学习问题，已展现出强大的实证性能。然而，目前尚不清楚基于PFN的因果估计器是否能提供与经典频率派估计器一致的不确定性量化。本研究通过分析基于PFN的平均处理效应（ATE）估计器的频率派一致性来填补这一空白。（1）我们证明，当将现有PFN解释为贝叶斯ATE估计器时，它们可能表现出先验诱导的混杂偏误：先验不会渐近地被数据覆盖，从而阻碍了频率派一致性。（2）作为补救措施，我们建议采用基于一步后验校正（OSPC）的校准程序。我们证明OSPC有助于恢复频率派一致性，并能为校准后的PFN（即校准后的基于PFN的估计器与经典半参数有效估计器随着数据量增大在分布上收敛）推导出半参数Bernstein-von Mises定理。（3）最后，我们在PFN之上通过定制鞅后验来实现OSPC。通过这种方式，我们能够从PFN中恢复OSPC所需的功能性干扰项后验。在多个（半）合成实验中，采用我们的鞅后验OSPC校准的PFN所产生的ATE不确定性（i）渐近匹配频率派不确定性，且（ii）与其他贝叶斯ATE估计器相比，在有限样本中具有良好校准性。