We consider the relative abilities and limitations of computationally efficient algorithms for learning in the presence of noise, under two well-studied and challenging adversarial noise models for learning Boolean functions: malicious noise, in which an adversary can arbitrarily corrupt a random subset of examples given to the learner; and nasty noise, in which an adversary can arbitrarily corrupt an adversarially chosen subset of examples given to the learner. We consider both the distribution-independent and fixed-distribution settings. Our main results highlight a dramatic difference between these two settings: For distribution-independent learning, we prove a strong equivalence between the two noise models: If a class ${\cal C}$ of functions is efficiently learnable in the presence of $η$-rate malicious noise, then it is also efficiently learnable in the presence of $η$-rate nasty noise. In sharp contrast, for the fixed-distribution setting we show an arbitrarily large separation: Under a standard cryptographic assumption, for any arbitrarily large value $r$ there exists a concept class for which there is a ratio of $r$ between the rate $η_{malicious}$ of malicious noise that polynomial-time learning algorithms can tolerate, versus the rate $η_{nasty}$ of nasty noise that such learning algorithms can tolerate. To offset the negative result for the fixed-distribution setting, we define a broad and natural class of algorithms, namely those that ignore contradictory examples (ICE). We show that for these algorithms, malicious noise and nasty noise are equivalent up to a factor of two in the noise rate: Any efficient ICE learner that succeeds with $η$-rate malicious noise can be converted to an efficient learner that succeeds with $η/2$-rate nasty noise. We further show that the above factor of two is necessary, again under a standard cryptographic assumption.

翻译：我们研究了在两种经过深入研究的、具有挑战性的对抗性噪声模型下，计算高效算法在噪声存在时学习布尔函数的相对能力和局限性：恶意噪声，其中对手可以任意破坏提供给学习者的随机示例子集；以及恶意噪声，其中对手可以任意破坏提供给学习者的、由对手选择的示例子集。我们考虑了分布无关和固定分布两种设置。我们的主要结果突显了这两种设置之间的显著差异：对于分布无关学习，我们证明了两种噪声模型之间的强等价性：如果函数类${\cal C}$在存在$η$速率恶意噪声的情况下可以有效学习，那么它在存在$η$速率恶意噪声的情况下也可以有效学习。与此形成鲜明对比的是，在固定分布设置中，我们展示了一个任意大的分离：在标准密码学假设下，对于任意大的值$r$，存在一个概念类，使得多项式时间学习算法能够容忍的恶意噪声速率$η_{malicious}$与这类学习算法能够容忍的恶意噪声速率$η_{nasty}$之间存在$r$倍的比率。为了抵消固定分布设置的负面结果，我们定义了一类广泛且自然的算法，即那些忽略矛盾示例（ICE）的算法。我们证明，对于这些算法，恶意噪声和恶意噪声在噪声速率上等价至多两倍：任何在$η$速率恶意噪声下成功的有效ICE学习者，都可以转换为在$η/2$速率恶意噪声下成功的有效学习者。我们进一步证明，在标准密码学假设下，上述两倍因子是必要的。