Generating calibrated and sharp neural network predictive distributions for regression problems is essential for optimal decision-making in many real-world applications. To address the miscalibration issue of neural networks, various methods have been proposed to improve calibration, including post-hoc methods that adjust predictions after training and regularization methods that act during training. While post-hoc methods have shown better improvement in calibration compared to regularization methods, the post-hoc step is completely independent of model training. We introduce a novel end-to-end model training procedure called Quantile Recalibration Training, integrating post-hoc calibration directly into the training process without additional parameters. We also present a unified algorithm that includes our method and other post-hoc and regularization methods, as particular cases. We demonstrate the performance of our method in a large-scale experiment involving 57 tabular regression datasets, showcasing improved predictive accuracy while maintaining calibration. We also conduct an ablation study to evaluate the significance of different components within our proposed method, as well as an in-depth analysis of the impact of the base model and different hyperparameters on predictive accuracy.

翻译：针对回归问题生成校准且尖锐的神经网络预测分布，对于许多实际应用中的最优决策至关重要。为解决神经网络的误校准问题，研究者提出了多种改进校准的方法，包括训练后调整预测的后期处理方法以及训练过程中起作用的正则化方法。尽管后期处理方法在校准改进方面优于正则化方法，但后期处理步骤完全独立于模型训练。我们引入了一种新颖的端到端模型训练流程——分位数重校准训练（Quantile Recalibration Training），该方法无需额外参数即可将后期处理校准直接集成到训练过程中。我们还提出了一种统一算法，将我们的方法与其他后期处理及正则化方法作为特例纳入其中。我们通过一项涉及57个表格回归数据集的大规模实验证明了该方法的性能，展示了在保持校准的同时提升预测精度。我们还进行了消融研究以评估所提方法中各组成部分的重要性，并深入分析了基础模型及不同超参数对预测精度的影响。