Probability predictions are essential to inform decision making across many fields. Ideally, probability predictions are (i) well calibrated, (ii) accurate, and (iii) bold, i.e., spread out enough to be informative for decision making. However, there is a fundamental tension between calibration and boldness, since calibration metrics can be high when predictions are overly cautious, i.e., non-bold. The purpose of this work is to develop a Bayesian model selection-based approach to assess calibration, and a strategy for boldness-recalibration that enables practitioners to responsibly embolden predictions subject to their required level of calibration. Specifically, we allow the user to pre-specify their desired posterior probability of calibration, then maximally embolden predictions subject to this constraint. We demonstrate the method with a case study on hockey home team win probabilities and then verify the performance of our procedures via simulation. We find that very slight relaxation of calibration probability (e.g., from 0.99 to 0.95) can often substantially embolden predictions when they are well calibrated and accurate (e.g., widening hockey predictions range from .26-.78 to .10-.91).

翻译：概率预测对于跨领域的决策制定至关重要。理想情况下，概率预测应具备（i）良好校准、（ii）准确性和（iii）胆量，即预测结果应足够分散以提供决策信息。然而，校准与胆量之间存在根本性矛盾，因为当预测过于保守（即缺乏胆量）时，校准指标可能仍然较高。本研究旨在开发一种基于贝叶斯模型选择的校准评估方法，并提出胆量重校准策略，使实践者能够在满足所需校准水平的前提下负责任地增强预测的胆量。具体而言，我们允许用户预设期望的后验校准概率，然后在此约束下最大化预测的胆量。我们通过冰球主场获胜概率的案例研究展示了该方法，并通过模拟实验验证了其性能。研究发现，当预测已具备良好校准和准确性时，略微放松校准概率（例如从0.99降至0.95）通常能显著增强预测的胆量（例如冰球预测范围从0.26-0.78扩大至0.10-0.91）。