While Convolutional Neural Networks (CNNs) excel at learning complex latent-space representations, their over-parameterization can lead to overfitting and reduced performance, particularly with limited data. This, alongside their high computational and memory demands, limits the applicability of CNNs for edge deployment. Low-rank matrix approximation has emerged as a promising approach to reduce CNN parameters, but its application presents challenges including rank selection and performance loss. To address these issues, we propose an efficient training method for CNN compression via dynamic parameter rank pruning. Our approach integrates efficient matrix factorization and novel regularization techniques, forming a robust framework for dynamic rank reduction and model compression. We use Singular Value Decomposition (SVD) to model low-rank convolutional filters and dense weight matrices and we achieve model compression by training the SVD factors with back-propagation in an end-to-end way. We evaluate our method on an array of modern CNNs, including ResNet-18, ResNet-20, and ResNet-32, and datasets like CIFAR-10, CIFAR-100, and ImageNet (2012), showcasing its applicability in computer vision. Our experiments show that the proposed method can yield substantial storage savings while maintaining or even enhancing classification performance.

翻译：尽管卷积神经网络（CNN）在学习复杂潜在空间表示方面表现出色，但其过度参数化可能导致过拟合及性能下降，尤其在数据有限的情况下。这一问题，加之其高昂的计算与内存需求，限制了CNN在边缘设备上的部署适用性。低秩矩阵近似已成为减少CNN参数的有效方法，但其应用仍面临秩选择及性能损失等挑战。为此，我们提出一种基于动态参数秩剪枝的高效CNN压缩训练方法。该方法融合了高效矩阵分解与新颖的正则化技术，构建了一个用于动态降秩与模型压缩的稳健框架。我们采用奇异值分解（SVD）对低秩卷积滤波器与稠密权重矩阵进行建模，并通过反向传播以端到端方式训练SVD因子实现模型压缩。我们在ResNet-18、ResNet-20、ResNet-32等现代CNN架构及CIFAR-10、CIFAR-100、ImageNet（2012）等数据集上评估该方法，验证其在计算机视觉中的适用性。实验表明，所提方法在维持甚至提升分类性能的同时，能显著节省存储空间。