Recent work has empirically shown that deep neural networks latch on to the Fourier statistics of training data and show increased sensitivity to Fourier-basis directions in the input. Understanding and modifying this Fourier-sensitivity of computer vision models may help improve their robustness. Hence, in this paper we study the frequency sensitivity characteristics of deep neural networks using a principled approach. We first propose a basis trick, proving that unitary transformations of the input-gradient of a function can be used to compute its gradient in the basis induced by the transformation. Using this result, we propose a general measure of any differentiable model's Fourier-sensitivity using the unitary Fourier-transform of its input-gradient. When applied to deep neural networks, we find that computer vision models are consistently sensitive to particular frequencies dependent on the dataset, training method and architecture. Based on this measure, we further propose a Fourier-regularization framework to modify the Fourier-sensitivities and frequency bias of models. Using our proposed regularizer-family, we demonstrate that deep neural networks obtain improved classification accuracy on robustness evaluations.

翻译：近期研究通过实验表明，深度神经网络会依赖训练数据的傅里叶统计特性，并对输入中傅里叶基方向表现出增强的敏感性。理解并修正计算机视觉模型的这种傅里叶敏感性，有助于提高其鲁棒性。为此，本文采用系统性方法研究深度神经网络的频率敏感性特征。我们首先提出基变换技巧，证明函数输入梯度的酉变换可用于计算该函数在变换所诱导基下的梯度。基于这一结果，我们利用输入梯度的酉傅里叶变换，提出一种适用于任意可微模型傅里叶敏感性的通用度量方法。将该方法应用于深度神经网络时发现，计算机视觉模型会持续对特定频率敏感，且这种敏感性取决于数据集、训练方法和网络架构。基于该度量方法，我们进一步提出傅里叶正则化框架，用于调整模型的傅里叶敏感性和频率偏好。实验表明，采用我们所提出的正则化函数族，深度神经网络在鲁棒性评估中的分类准确率得到显著提升。