Uncertainty is a fundamental aspect of real-world scenarios, where perfect information is rarely available. Humans naturally develop complex internal models to navigate incomplete data and effectively respond to unforeseen or partially observed events. In machine learning, Focal Loss is commonly used to reduce misclassification rates by emphasizing hard-to-classify samples. However, it does not guarantee well-calibrated predicted probabilities and may result in models that are overconfident or underconfident. High calibration error indicates a misalignment between predicted probabilities and actual outcomes, affecting model reliability. This research introduces a novel loss function called Focal Calibration Loss (FCL), designed to improve probability calibration while retaining the advantages of Focal Loss in handling difficult samples. By minimizing the Euclidean norm through a strictly proper loss, FCL penalizes the instance-wise calibration error and constrains bounds. We provide theoretical validation for proposed method and apply it to calibrate CheXNet for potential deployment in web-based health-care systems. Extensive evaluations on various models and datasets demonstrate that our method achieves SOTA performance in both calibration and accuracy metrics.

翻译：不确定性是现实世界场景的基本特征，完美信息几乎无法获取。人类自然地发展出复杂的内部模型以应对不完整数据，并对未预见或部分观测的事件作出有效响应。在机器学习中，Focal Loss 通常通过强调难以分类的样本来降低误分类率。然而，它并不能保证得到良好校准的预测概率，可能导致模型过度自信或自信不足。高校准误差表明预测概率与实际结果之间存在偏差，影响模型可靠性。本研究提出了一种称为 Focal Calibration Loss (FCL) 的新型损失函数，旨在改善概率校准，同时保留 Focal Loss 在处理困难样本方面的优势。通过严格适当损失函数最小化欧氏范数，FCL 惩罚实例级校准误差并约束边界。我们为所提方法提供了理论验证，并将其应用于校准 CheXNet，以实现在基于网络的医疗系统中的潜在部署。在各种模型和数据集上的广泛评估表明，我们的方法在校准和准确率指标上均达到了 SOTA 性能。