Calibration of neural networks is a topical problem that is becoming more and more important as neural networks increasingly underpin real-world applications. The problem is especially noticeable when using modern neural networks, for which there is a significant difference between the confidence of the model and the probability of correct prediction. Various strategies have been proposed to improve calibration, yet accurate calibration remains challenging. We propose a novel framework with two contributions: introducing a new differentiable surrogate for expected calibration error (DECE) that allows calibration quality to be directly optimised, and a meta-learning framework that uses DECE to optimise for validation set calibration with respect to model hyper-parameters. The results show that we achieve competitive performance with existing calibration approaches. Our framework opens up a new avenue and toolset for tackling calibration, which we believe will inspire further work on this important challenge.

翻译：神经网络校准是一个热点问题，随着神经网络日益成为现实应用的基础，这一问题的重要性与日俱增。当使用现代神经网络时，该问题尤为突出，因为模型置信度与正确预测概率之间存在显著差异。尽管已有多种策略被提出以改进校准，但实现精准校准仍具挑战性。我们提出了一种新颖框架，包含两项贡献：引入一种新的可微期望校准误差代理（DECE），使得校准质量能够直接优化；以及一个元学习框架，该框架利用DECE针对模型超参数优化验证集校准效果。结果表明，我们达到了与现有校准方法相竞争的性能。我们的框架为解决校准问题开辟了新途径并提供了新工具集，我们相信这将为这一重要挑战的研究注入新动力。