Despite the undeniable progress in visual recognition tasks fueled by deep neural networks, there exists recent evidence showing that these models are poorly calibrated, resulting in over-confident predictions. The standard practices of minimizing the cross entropy loss during training promote the predicted softmax probabilities to match the one-hot label assignments. Nevertheless, this yields a pre-softmax activation of the correct class that is significantly larger than the remaining activations, which exacerbates the miscalibration problem. Recent observations from the classification literature suggest that loss functions that embed implicit or explicit maximization of the entropy of predictions yield state-of-the-art calibration performances. Despite these findings, the impact of these losses in the relevant task of calibrating medical image segmentation networks remains unexplored. In this work, we provide a unifying constrained-optimization perspective of current state-of-the-art calibration losses. Specifically, these losses could be viewed as approximations of a linear penalty (or a Lagrangian term) imposing equality constraints on logit distances. This points to an important limitation of such underlying equality constraints, whose ensuing gradients constantly push towards a non-informative solution, which might prevent from reaching the best compromise between the discriminative performance and calibration of the model during gradient-based optimization. Following our observations, we propose a simple and flexible generalization based on inequality constraints, which imposes a controllable margin on logit distances. Comprehensive experiments on a variety of public medical image segmentation benchmarks demonstrate that our method sets novel state-of-the-art results on these tasks in terms of network calibration, whereas the discriminative performance is also improved.

翻译：尽管深度神经网络推动了视觉识别任务的显著进展，但近期有证据表明这些模型校准不良，导致预测过度自信。在训练过程中最小化交叉熵损失的标准做法促使预测的softmax概率与独热标签分配相匹配。然而，这导致正确类别的预激活值显著高于其他激活值，从而加剧了未校准问题。分类文献的最新观测表明，嵌入隐式或显式预测熵最大化的损失函数可实现最优的校准性能。尽管有这些发现，但这些损失在医学图像分割网络校准这一相关任务中的影响仍未被探索。在本工作中，我们提供了一个统一的约束优化视角来解释当前最先进的校准损失。具体而言，这些损失可视为对logit距离施加等式约束的线性惩罚（或拉格朗日项）的近似。这揭示了此类等式约束的重要局限性——其后续梯度持续推动非信息性解，可能在基于梯度的优化过程中阻碍模型在判别性能与校准之间达到最佳平衡。基于这一发现，我们提出了一种基于不等式约束的简单灵活的泛化方法，对logit距离施加可控边际。在多个公开医学图像分割基准上的全面实验表明，我们的方法在网络校准方面创造了新的最先进结果，同时判别性能也得到了提升。