Uncertainty quantification is a pivotal field that contributes to the realization of reliable and robust systems. By providing complementary information, it becomes instrumental in fortifying safe decisions, particularly within high-risk applications. Nevertheless, a comprehensive understanding of the advantages and limitations inherent in various methods within the medical imaging field necessitates further research coupled with in-depth analysis. In this paper, we explore Conformal Prediction, an emerging distribution-free uncertainty quantification technique, along with Monte Carlo Dropout and Evidential Deep Learning methods. Our comprehensive experiments provide a comparative performance analysis for skin lesion classification tasks across the three quantification methods. Furthermore, We present insights into the effectiveness of each method in handling Out-of-Distribution samples from domain-shifted datasets. Based on our experimental findings, our conclusion highlights the robustness and consistent performance of conformal prediction across diverse conditions. This positions it as the preferred choice for decision-making in safety-critical applications.

翻译：不确定性量化是实现可靠和稳健系统的关键领域。通过提供补充信息，它在加强安全决策方面发挥着重要作用，尤其是在高风险应用中。然而，全面理解医学成像领域中各种方法的优势和局限性仍需进一步研究及深入分析。本文探讨了新兴的无分布假设不确定性量化技术——Conformal预测，以及蒙特卡洛Dropout和证据深度学习。我们通过全面实验，针对皮肤病变分类任务对三种量化方法进行了比较性能分析。此外，我们揭示了每种方法在处理来自域偏移数据集的分布外样本时的有效性。基于实验结果，我们得出Conformal预测在不同条件下具有鲁棒性和一致性能的结论，使其成为安全关键应用中决策制定的首选方法。