Estimates of causal parameters such as conditional average treatment effects and conditional quantile treatment effects play an important role in real-world decision making. Given this importance, one should ensure these estimators are calibrated. While there is a rich literature on calibrating estimators of non-causal parameters, very few methods have been derived for calibrating estimators of causal parameters, or more generally estimators of quantities involving nuisance parameters. In this work, we provide a general framework for calibrating predictors involving nuisance estimation. We consider a notion of calibration defined with respect to an arbitrary, nuisance-dependent loss $\ell$, under which we say an estimator $\theta$ is calibrated if its predictions cannot be changed on any level set to decrease loss. We prove generic upper bounds on the calibration error of any causal parameter estimate $\theta$ with respect to any loss $\ell$ using a concept called Neyman Orthogonality. Our bounds involve two decoupled terms - one measuring the error in estimating the unknown nuisance parameters, and the other representing the calibration error in a hypothetical world where the learned nuisance estimates were true. We use our bound to analyze the convergence of two sample splitting algorithms for causal calibration. One algorithm, which applies to universally orthogonalizable loss functions, transforms the data into generalized pseudo-outcomes and applies an off-the-shelf calibration procedure. The other algorithm, which applies to conditionally orthogonalizable loss functions, extends the classical uniform mass binning algorithm to include nuisance estimation. Our results are exceedingly general, showing that essentially any existing calibration algorithm can be used in causal settings, with additional loss only arising from errors in nuisance estimation.

翻译：条件平均处理效应和条件分位数处理效应等因果参数的估计在现实世界决策中起着重要作用。鉴于其重要性，应确保这些估计量经过校准。尽管现有文献对非因果参数估计量的校准已有丰富研究，但针对因果参数估计量或更广义上涉及干扰参数估计量的校准方法却极少。本研究提出了一个用于校准涉及干扰估计的预测器的一般框架。我们考虑一种基于任意干扰依赖损失函数$\ell$定义的校准概念：若估计量$\theta$的预测在任何水平集上都无法通过改变来降低损失，则称其已校准。我们利用名为奈曼正交性的概念，针对任意损失函数$\ell$证明了任意因果参数估计$\theta$的校准误差的通用上界。所得上界包含两个解耦项：一项衡量未知干扰参数估计的误差，另一项表示在假设已学干扰估计为真实情况下的校准误差。我们运用该界分析两种样本分割算法在因果校准中的收敛性。第一种算法适用于普遍可正交化损失函数，通过将数据转换为广义伪结果并应用现成的校准程序实现。第二种算法适用于条件可正交化损失函数，将经典均匀质量分箱算法扩展至包含干扰估计的场景。我们的结果具有极强普适性，表明几乎所有现有校准算法均可应用于因果场景，且额外损失仅来源于干扰估计误差。