Deep neural networks are vulnerable to universal adversarial perturbation (UAP), an instance-agnostic perturbation capable of fooling the target model for most samples. Compared to instance-specific adversarial examples, UAP is more challenging as it needs to generalize across various samples and models. In this paper, we examine the serious dilemma of UAP generation methods from a generalization perspective -- the gradient vanishing problem using small-batch stochastic gradient optimization and the local optima problem using large-batch optimization. To address these problems, we propose a simple and effective method called Stochastic Gradient Aggregation (SGA), which alleviates the gradient vanishing and escapes from poor local optima at the same time. Specifically, SGA employs the small-batch training to perform multiple iterations of inner pre-search. Then, all the inner gradients are aggregated as a one-step gradient estimation to enhance the gradient stability and reduce quantization errors. Extensive experiments on the standard ImageNet dataset demonstrate that our method significantly enhances the generalization ability of UAP and outperforms other state-of-the-art methods. The code is available at https://github.com/liuxuannan/Stochastic-Gradient-Aggregation.

翻译：深度神经网络容易受到通用对抗扰动（UAP）的攻击，这是一种与实例无关的扰动，能够使目标模型在大多数样本上失效。与针对特定实例的对抗样本相比，UAP更具挑战性，因为它需要在不同样本和模型之间实现泛化。本文从泛化角度审视UAP生成方法面临的严重困境——小批量随机梯度优化中的梯度消失问题和大批量优化中的局部最优问题。为了解决这些问题，我们提出了一种简单有效的方法，称为随机梯度聚合（SGA），该方法能够同时缓解梯度消失并跳出不良局部最优。具体而言，SGA采用小批量训练进行多次迭代的内部预搜索，然后将所有内部梯度聚合为一步梯度估计，以增强梯度稳定性并降低量化误差。在标准ImageNet数据集上的大量实验表明，我们的方法显著增强了UAP的泛化能力，并优于其他最先进的方法。代码可在 https://github.com/liuxuannan/Stochastic-Gradient-Aggregation 获取。