The rapid advances in Vision Transformer (ViT) refresh the state-of-the-art performances in various vision tasks, overshadowing the conventional CNN-based models. This ignites a few recent striking-back research in the CNN world showing that pure CNN models can achieve as good performance as ViT models when carefully tuned. While encouraging, designing such high-performance CNN models is challenging, requiring non-trivial prior knowledge of network design. To this end, a novel framework termed Mathematical Architecture Design for Deep CNN (DeepMAD) is proposed to design high-performance CNN models in a principled way. In DeepMAD, a CNN network is modeled as an information processing system whose expressiveness and effectiveness can be analytically formulated by their structural parameters. Then a constrained mathematical programming (MP) problem is proposed to optimize these structural parameters. The MP problem can be easily solved by off-the-shelf MP solvers on CPUs with a small memory footprint. In addition, DeepMAD is a pure mathematical framework: no GPU or training data is required during network design. The superiority of DeepMAD is validated on multiple large-scale computer vision benchmark datasets. Notably on ImageNet-1k, only using conventional convolutional layers, DeepMAD achieves 0.7% and 1.5% higher top-1 accuracy than ConvNeXt and Swin on Tiny level, and 0.8% and 0.9% higher on Small level.

翻译：视觉Transformer（Vision Transformer, ViT）的快速发展刷新了各类视觉任务的最优性能，使传统的基于卷积神经网络（CNN）的模型相形见绌。这激发了CNN领域近期一些引人注目的反击研究，表明精心调优的纯CNN模型能够达到与ViT模型相当的性能。尽管这一发现令人鼓舞，但设计此类高性能CNN模型仍然充满挑战，需要对网络设计具备深厚的先验知识。为此，本文提出一种名为深度CNN数学架构设计（DeepMAD）的新框架，以系统化的方式设计高性能CNN模型。在DeepMAD中，CNN网络被建模为一个信息处理系统，其表达能力和有效性可通过结构参数进行解析公式化。随后，提出一个带约束的数学规划问题来优化这些结构参数。该数学规划问题可利用现成的数学规划求解器在CPU上轻松求解，且内存占用极小。此外，DeepMAD是一个纯数学框架：网络设计过程中无需GPU或训练数据。DeepMAD的优越性在多个大规模计算机视觉基准数据集上得到了验证。值得注意的是，在ImageNet-1k上，仅使用传统卷积层，DeepMAD在Tiny级别相较于ConvNeXt和Swin分别实现了0.7%和1.5%的top-1准确率提升，在Small级别则分别提升了0.8%和0.9%。