The classic concept of "calibrated forecasts" and its more recent refinement, "calibeating," are defined with respect to the standard quadratic scoring rule. We extend these notions to the class of $\textit{proper}$ scoring rules (for which the best forecast is the true distribution) and define $\textit{proper-calibration}$ and $\textit{proper-calibeating}$ by requiring the errors to converge to zero uniformly over all bounded proper scoring rules. We first establish that calibration always implies proper-calibration, whereas calibeating need not imply proper-calibeating. Second, we show how to guarantee proper-calibeating and proper-multicalibeating. Finally, we demonstrate the equivalence between proper-calibration and universal no regret when best replying to forecasts in decision-making under uncertainty.

翻译：经典的“校准预测”概念及其近期改进的“校准评估”是针对标准二次评分规则定义的。我们将这些概念推广到$\textit{严格合理的}$评分规则类别（对此类规则，最优预测为真实分布），并通过要求误差在所有有界严格合理评分规则上一致收敛到零，来定义$\textit{严格合理校准}$和$\textit{严格合理校准评估}$。我们首先证明：校准总能推出严格合理校准，而校准评估未必推出严格合理校准评估。其次，我们展示了如何保证严格合理校准评估和严格合理多重校准评估。最后，我们证明了在不确定性决策中，当针对预测进行最优应对时，严格合理校准与普遍无悔之间的等价性。