The classic concept of "calibrated forecasts" and its more recent refinement, "calibeating," are defined with respect to the standard quadratic scoring rule. We extend these notions to the class of $\textit{proper}$ scoring rules (for which the best forecast is the true distribution) and define $\textit{proper-calibration}$ and $\textit{proper-calibeating}$ by requiring the errors to converge to zero uniformly over all bounded proper scoring rules. We first establish that calibration always implies proper-calibration, whereas calibeating need not imply proper-calibeating. Second, we show how to guarantee proper-calibeating and proper-multicalibeating. Finally, we demonstrate the equivalence between proper-calibration and universal no regret when best replying to forecasts in decision-making under uncertainty.

翻译：“校准预测”（calibrated forecasts）的经典概念及其近期改进“校准节拍”（calibeating）通常基于标准二次评分规则进行定义。我们将这些概念扩展到$\textit{真确}$（proper）评分规则类（在此规则下，最优预测即为真实分布），并通过要求误差在所有有界真确评分规则上一致收敛至零，进而定义“真确校准”（proper-calibration）与“真确校准节拍”（proper-calibeating）。我们首先证明：校准必然蕴含真确校准，而校准节拍不一定蕴含真确校准节拍。其次，我们展示了如何确保真确校准节拍与真确多重校准节拍（proper-multicalibeating）。最后，我们证明了在不确定性决策中对预测进行最优响应时，真确校准与无遗憾（universal no regret）之间的等价性。