Convolutional neural networks have demonstrated impressive results in many computer vision tasks. However, the increasing size of these networks raises concerns about the information overload resulting from the large number of network parameters. In this paper, we propose Frequency Regularization to restrict the non-zero elements of the network parameters in the frequency domain. The proposed approach operates at the tensor level, and can be applied to almost all network architectures. Specifically, the tensors of parameters are maintained in the frequency domain, where high frequency components can be eliminated by zigzag setting tensor elements to zero. Then, the inverse discrete cosine transform (IDCT) is used to reconstruct the spatial tensors for matrix operations during network training. Since high frequency components of images are known to be less critical, a large proportion of these parameters can be set to zero when networks are trained with the proposed frequency regularization. Comprehensive evaluations on various state-of-the-art network architectures, including LeNet, Alexnet, VGG, Resnet, ViT, UNet, GAN, and VAE, demonstrate the effectiveness of the proposed frequency regularization. For a very small accuracy decrease (less than 2\%), a LeNet5 with 0.4M parameters can be represented by only 776 float16 numbers (over 1100$\times$ reduction), and a UNet with 34M parameters can be represented by only 759 float16 numbers (over 80000$\times$ reduction). In particular, the original size of the UNet model is 366MB, we reduce it to 4.5kb.

翻译：卷积神经网络在许多计算机视觉任务中取得了令人瞩目的成果。然而，这些网络规模的不断扩大引发了人们对大量网络参数导致信息过载的担忧。本文提出频率正则化方法，通过在频域中限制网络参数的非零元素。所提出的方法在张量级别上运行，可应用于几乎所有网络架构。具体而言，参数张量在频域中维护，通过锯齿形将张量元素置零来消除高频分量。随后，利用逆离散余弦变换（IDCT）重构空间张量，以进行网络训练期间的矩阵运算。由于图像的高频分量已知不那么关键，在使用所提出的频率正则化训练网络时，这些参数中的很大比例可以被设为零。在多种最先进的网络架构上进行的全面评估，包括LeNet、Alexnet、VGG、Resnet、ViT、UNet、GAN和VAE，证明了所提出频率正则化的有效性。在精度下降极小（小于2%）的情况下，一个具有0.4M参数的LeNet5仅需776个float16数即可表示（压缩超过1100倍），而一个具有34M参数的UNet仅需759个float16数即可表示（压缩超过80000倍）。特别是，UNet模型的原始大小为366MB，我们将其缩减至4.5kb。