We study the robustness of conformal prediction, a powerful tool for uncertainty quantification, to label noise. Our analysis tackles both regression and classification problems, characterizing when and how it is possible to construct uncertainty sets that correctly cover the unobserved noiseless ground truth labels. We further extend our theory and formulate the requirements for correctly controlling a general loss function, such as the false negative proportion, with noisy labels. Our theory and experiments suggest that conformal prediction and risk-controlling techniques with noisy labels attain conservative risk over the clean ground truth labels except in adversarial cases. In such cases, we can also correct for noise of bounded size in the conformal prediction algorithm in order to ensure achieving the correct risk of the ground truth labels without score or data regularity.
翻译:我们研究了置信预测(一种强大的不确定性量化工具)对标签噪声的鲁棒性。本分析涵盖回归与分类问题,揭示了在何种条件下以及如何构建能正确覆盖未观测无噪声真实标签的不确定性集。我们进一步扩展理论,阐明了利用带噪标签正确控制一般损失函数(如假阴性比例)所需的条件。理论与实验表明,除对抗性情形外,采用带噪标签的置信预测与风险控制技术可在清洁真实标签上实现保守风险。在对抗情形中,我们还可通过修正置信预测算法中有界大小的噪声,确保无需分数或数据正则性即可获得真实标签的正确风险。