We initiate the study of tolerant adversarial PAC-learning with respect to metric perturbation sets. In adversarial PAC-learning, an adversary is allowed to replace a test point $x$ with an arbitrary point in a closed ball of radius $r$ centered at $x$. In the tolerant version, the error of the learner is compared with the best achievable error with respect to a slightly larger perturbation radius $(1+\gamma)r$. This simple tweak helps us bridge the gap between theory and practice and obtain the first PAC-type guarantees for algorithmic techniques that are popular in practice. Our first result concerns the widely-used ``perturb-and-smooth'' approach for adversarial learning. For perturbation sets with doubling dimension $d$, we show that a variant of these approaches PAC-learns any hypothesis class $\mathcal{H}$ with VC-dimension $v$ in the $\gamma$-tolerant adversarial setting with $O\left(\frac{v(1+1/\gamma)^{O(d)}}{\varepsilon}\right)$ samples. This is in contrast to the traditional (non-tolerant) setting in which, as we show, the perturb-and-smooth approach can provably fail. Our second result shows that one can PAC-learn the same class using $\widetilde{O}\left(\frac{d.v\log(1+1/\gamma)}{\varepsilon^2}\right)$ samples even in the agnostic setting. This result is based on a novel compression-based algorithm, and achieves a linear dependence on the doubling dimension as well as the VC-dimension. This is in contrast to the non-tolerant setting where there is no known sample complexity upper bound that depend polynomially on the VC-dimension.

翻译：我们首次研究了关于度量扰动集的容忍性对抗PAC学习。在对抗PAC学习中，对手允许用半径为$r$的闭球内任意点替换测试点$x$。在容忍性版本中，学习器的误差与使用稍大扰动半径$(1+\gamma)r$所能达到的最佳误差进行比较。这一简单调整有助于弥合理论与实践的差距，并为实践中流行的算法技术提供首个PAC类型保证。我们的第一个结果涉及广泛使用的用于对抗学习的"扰动-平滑"方法。对于倍数为$d$的扰动集，我们证明这些方法的变体能够以$O\left(\frac{v(1+1/\gamma)^{O(d)}}{\varepsilon}\right)$个样本在$\gamma$-容忍对抗设置中PAC学习任何VC维为$v$的假设类$\mathcal{H}$。这与传统（非容忍）设置形成对比，我们证明在该设置中扰动-平滑方法可能失效。我们的第二个结果表明，即使在不可知设置中，也可以使用$\widetilde{O}\left(\frac{d\cdot v\log(1+1/\gamma)}{\varepsilon^2}\right)$个样本PAC学习同一类。该结果基于一种新颖的基于压缩的算法，实现了对倍数维和VC维的线性依赖。这不同于非容忍设置，在该设置中不存在已知的依赖于VC维多项式的样本复杂度上界。