Recently, lightweight Vision Transformers (ViTs) demonstrate superior performance and lower latency compared with lightweight Convolutional Neural Networks (CNNs) on resource-constrained mobile devices. This improvement is usually attributed to the multi-head self-attention module, which enables the model to learn global representations. However, the architectural disparities between lightweight ViTs and lightweight CNNs have not been adequately examined. In this study, we revisit the efficient design of lightweight CNNs and emphasize their potential for mobile devices. We incrementally enhance the mobile-friendliness of a standard lightweight CNN, specifically MobileNetV3, by integrating the efficient architectural choices of lightweight ViTs. This ends up with a new family of pure lightweight CNNs, namely RepViT. Extensive experiments show that RepViT outperforms existing state-of-the-art lightweight ViTs and exhibits favorable latency in various vision tasks. On ImageNet, RepViT achieves over 80\% top-1 accuracy with nearly 1ms latency on an iPhone 12, which is the first time for a lightweight model, to the best of our knowledge. Our largest model, RepViT-M3, obtains 81.4\% accuracy with only 1.3ms latency. The code and trained models are available at \url{https://github.com/jameslahm/RepViT}.

翻译：近期，轻量级视觉Transformer（ViT）在资源受限的移动设备上展现出相较于轻量级卷积神经网络（CNN）更优的性能与更低的延迟。这一提升通常归因于多头自注意力模块，该模块使模型能够学习全局表征。然而，轻量级ViT与轻量级CNN之间的架构差异尚未得到充分审视。在本研究中，我们重新探讨了轻量级CNN的高效设计，并强调其在移动设备上的潜力。我们通过整合轻量级ViT的高效架构选择，逐步提升了标准轻量级CNN（具体为MobileNetV3）的移动端友好性，最终衍生出一系列纯轻量级CNN新模型，即RepViT。大量实验表明，RepViT在多种视觉任务中超越了现有最先进的轻量级ViT，并展现出优异的延迟表现。在ImageNet上，RepViT在iPhone 12上以近1ms的延迟实现了超过80%的Top-1准确率——据我们所知，这是轻量级模型首次达到这一水平。我们的最大模型RepViT-M3仅以1.3ms延迟便获得了81.4%的准确率。代码与预训练模型已开源至 \url{https://github.com/jameslahm/RepViT}。