Accurate classification requires not only high predictive accuracy but also well-calibrated confidence estimates. Yet, modern deep neural networks (DNNs) are often overconfident, primarily due to overfitting on the negative log-likelihood (NLL). While focal loss variants alleviate this issue, they typically reduce accuracy, revealing a persistent trade-off between calibration and predictive performance. Motivated by the complementary strengths of generative and discriminative classifiers, we propose Generative Cross-Entropy (GCE), which maximizes $p(x|y)$ and is equivalent to cross-entropy augmented with a class-level confidence regularizer. Under mild conditions, GCE is strictly proper. Across CIFAR-10/100, Tiny-ImageNet, and a medical imaging benchmark, GCE improves both accuracy and calibration over cross-entropy, especially in the long-tailed scenario. Combined with adaptive piecewise temperature scaling (ATS), GCE attains calibration competitive with focal-loss variants without sacrificing accuracy.
翻译:精确分类不仅要求高预测精度,还需良好校准的置信度估计。然而,现代深度神经网络(DNN)常因过度拟合负对数似然(NLL)而表现出过度自信。尽管焦点损失变体可缓解此问题,但通常会降低精度,揭示校准与预测性能间固有的权衡。受生成式与判别式分类器互补优势的启发,我们提出生成式交叉熵(GCE),该方法最大化 $p(x|y)$,等价于引入类别级置信度正则化的交叉熵。在温和条件下,GCE严格适定。在CIFAR-10/100、Tiny-ImageNet及医学影像基准实验中,GCE在长尾场景下尤其能同时提升交叉熵的精度与校准性能。结合自适应分段温度缩放(ATS),GCE在保持精度的同时实现了与焦点损失变体竞争的校准效果。