One of the remarkable properties of robust computer vision models is that their input-gradients are often aligned with human perception, referred to in the literature as perceptually-aligned gradients (PAGs). Despite only being trained for classification, PAGs cause robust models to have rudimentary generative capabilities, including image generation, denoising, and in-painting. However, the underlying mechanisms behind these phenomena remain unknown. In this work, we provide a first explanation of PAGs via \emph{off-manifold robustness}, which states that models must be more robust off- the data manifold than they are on-manifold. We first demonstrate theoretically that off-manifold robustness leads input gradients to lie approximately on the data manifold, explaining their perceptual alignment. We then show that Bayes optimal models satisfy off-manifold robustness, and confirm the same empirically for robust models trained via gradient norm regularization, noise augmentation, and randomized smoothing. Quantifying the perceptual alignment of model gradients via their similarity with the gradients of generative models, we show that off-manifold robustness correlates well with perceptual alignment. Finally, based on the levels of on- and off-manifold robustness, we identify three different regimes of robustness that affect both perceptual alignment and model accuracy: weak robustness, bayes-aligned robustness, and excessive robustness.

翻译：鲁棒计算机视觉模型的一个显著特性是其输入梯度常与人类感知对齐，文献中称之为感知对齐梯度（PAGs）。尽管仅针对分类任务训练，PAGs赋予了鲁棒模型基础的生成能力，如图像生成、去噪和修复。然而，这些现象背后的机制仍不明确。本文通过“流形外鲁棒性”首次解释了PAGs，该理论要求模型在数据流形外比流形内具有更强的鲁棒性。我们首先从理论上证明，流形外鲁棒性导致输入梯度近似位于数据流形上，从而解释其感知对齐性。随后表明贝叶斯最优模型满足流形外鲁棒性，并通过梯度范数正则化、噪声增强和随机平滑训练的鲁棒模型在实证上验证了这一点。通过比较模型梯度与生成模型梯度的相似性来定量评估感知对齐程度，我们发现流形外鲁棒性与感知对齐显著相关。最后，基于流形内与流形外鲁棒性的差异，我们识别出三种影响感知对齐和模型准确率的鲁棒性机制：弱鲁棒性、贝叶斯对齐鲁棒性和过度鲁棒性。