We offer a study that connects robust discriminative classifiers trained with adversarial training (AT) with generative modeling in the form of Energy-based Models (EBM). We do so by decomposing the loss of a discriminative classifier and showing that the discriminative model is also aware of the input data density. Though a common assumption is that adversarial points leave the manifold of the input data, our study finds out that, surprisingly, untargeted adversarial points in the input space are very likely under the generative model hidden inside the discriminative classifier -- have low energy in the EBM. We present two evidence: untargeted attacks are even more likely than the natural data and their likelihood increases as the attack strength increases. This allows us to easily detect them and craft a novel attack called High-Energy PGD that fools the classifier yet has energy similar to the data set. The code is available at github.com/senad96/Robust-Generative

翻译：我们开展了一项研究，将经过对抗训练（AT）的鲁棒判别分类器与能量基模型（EBM）形式的生成建模联系起来。为此，我们对判别分类器的损失函数进行分解，并证明判别模型也能感知输入数据的密度。尽管通常认为对抗样本会偏离输入数据流形，但我们的研究发现，令人惊讶的是，输入空间中非目标对抗样本在判别分类器内隐含的生成模型下具有很高的似然性——即在EBM中具有低能量。我们提供了两项证据：非目标攻击的似然性甚至高于自然数据，且其似然性随攻击强度增加而增大。这使我们能够轻松检测此类攻击，并设计出一种名为高能PGD的新型攻击方法，该方法既能欺骗分类器，又能保持与数据集相近的能量。代码已开源至github.com/senad96/Robust-Generative。