Learning with label dependent label noise has been extensively explored in both theory and practice; however, dealing with instance (i.e., feature) and label dependent label noise continues to be a challenging task. The difficulty arises from the fact that the noise rate varies for each instance, making it challenging to estimate accurately. The question of whether it is possible to learn a reliable model using only noisy samples remains unresolved. We answer this question with a theoretical analysis that provides matching upper and lower bounds. Surprisingly, our results show that, without any additional assumptions, empirical risk minimization achieves the optimal excess risk bound. Specifically, we derive a novel excess risk bound proportional to the noise level, which holds in very general settings, by comparing the empirical risk minimizers obtained from clean samples and noisy samples. Second, we show that the minimax lower bound for the 0-1 loss is a constant proportional to the average noise rate. Our findings suggest that learning solely with noisy samples is impossible without access to clean samples or strong assumptions on the distribution of the data.

翻译：在理论和实践中，标签依赖型标签噪声学习已被广泛探讨；然而，处理实例（即特征）与标签依赖的标签噪声仍是一项挑战。其难点在于每个实例的噪声率各不相同，导致难以准确估计。仅使用含噪样本能否学习到可靠模型的问题尚未解决。我们通过理论分析回答了这个问题，给出了匹配的上界与下界。令人意外的是，我们的结果表明，在无任何额外假设的情况下，经验风险最小化能达到最优超额风险界。具体而言，通过比较从干净样本和含噪样本获得的经验风险最小化器，我们推导出一种与噪声水平成比例的新型超额风险界，该界在非常一般的条件下成立。其次，我们证明了0-1损失的极小化最大下界是与平均噪声率成比例的常数。我们的发现表明，若无干净样本或对数据分布施加强假设，仅依靠含噪样本进行学习是不可能的。