Moderate calibration, the expected event probability among observations with predicted probability z being equal to z, is a desired property of risk prediction models. Current graphical and numerical techniques for evaluating moderate calibration of risk prediction models are mostly based on smoothing or grouping the data. As well, there is no widely accepted inferential method for the null hypothesis that a model is moderately calibrated. In this work, we discuss recently-developed, and propose novel, methods for the assessment of moderate calibration for binary responses. The methods are based on the limiting distributions of functions of standardized partial sums of prediction errors converging to the corresponding laws of Brownian motion. The novel method relies on well-known properties of the Brownian bridge which enables joint inference on mean and moderate calibration, leading to a unified "bridge" test for detecting miscalibration. Simulation studies indicate that the bridge test is more powerful, often substantially, than the alternative test. As a case study we consider a prediction model for short-term mortality after a heart attack, where we provide suggestions on graphical presentation and the interpretation of results. Moderate calibration can be assessed without requiring arbitrary grouping of data or using methods that require tuning of parameters. An accompanying R package implements this method (see https://github.com/resplab/cumulcalib/).

翻译：适度校准是指观测值中预测概率为z时的事件期望概率等于z，这是风险预测模型期望具备的性质。当前评估风险预测模型适度校准的图形和数值方法大多基于数据平滑或分组处理。此外，对于模型具有适度校准性的零假设，目前尚未形成广泛接受的推断方法。本研究讨论了近期发展的方法，并提出了针对二元响应变量适度校准评估的新方法。这些方法基于预测误差标准化部分和函数的极限分布收敛于布朗运动的相应规律。新方法利用布朗桥的已知特性，能够对均值与适度校准进行联合推断，从而形成统一的"桥"检验来检测错误校准。模拟研究表明，桥检验比替代检验方法具有更高的检验功效，且往往显著提升。通过急性心肌梗死后短期死亡率预测模型的案例研究，我们提供了图形展示与结果解读的建议。适度校准评估无需对数据进行任意分组，也无需使用需要参数调优的方法。配套的R包已实现该方法（参见https://github.com/resplab/cumulcalib/）。