Recently, lightweight Vision Transformers (ViTs) demonstrate superior performance and lower latency compared with lightweight Convolutional Neural Networks (CNNs) on resource-constrained mobile devices. This improvement is usually attributed to the multi-head self-attention module, which enables the model to learn global representations. However, the architectural disparities between lightweight ViTs and lightweight CNNs have not been adequately examined. In this study, we revisit the efficient design of lightweight CNNs and emphasize their potential for mobile devices. We incrementally enhance the mobile-friendliness of a standard lightweight CNN, specifically MobileNetV3, by integrating the efficient architectural choices of lightweight ViTs. This ends up with a new family of pure lightweight CNNs, namely RepViT. Extensive experiments show that RepViT outperforms existing state-of-the-art lightweight ViTs and exhibits favorable latency in various vision tasks. On ImageNet, RepViT achieves over 80\% top-1 accuracy with nearly 1ms latency on an iPhone 12, which is the first time for a lightweight model, to the best of our knowledge. Our largest model, RepViT-M3, obtains 81.4\% accuracy with only 1.3ms latency. The code and trained models are available at \url{https://github.com/jameslahm/RepViT}.

翻译：近期，轻量级视觉Transformer（ViT）在资源受限的移动设备上展现出优于轻量级卷积神经网络（CNN）的性能与更低延迟。这一改进通常归因于使模型能够学习全局表征的多头自注意力模块。然而，轻量级ViT与轻量级CNN之间的架构差异尚未得到充分探讨。在本研究中，我们重新审视了轻量级CNN的高效设计，并着重强调其在移动设备上的潜力。我们通过整合轻量级ViT的高效架构选择，逐步增强标准轻量级CNN（特别是MobileNetV3）的移动端友好性，最终衍生出一个全新的纯轻量级CNN系列——RepViT。大量实验表明，RepViT在各类视觉任务中均优于现有最先进的轻量级ViT，并展现出优越的延迟表现。在ImageNet上，RepViT在iPhone 12上以近1ms延迟实现超过80%的Top-1准确率，据我们所知，这是轻量级模型首次达到这一水平。我们最大的模型RepViT-M3仅需1.3ms延迟便可获得81.4%的准确率。代码与预训练模型已开源至\url{https://github.com/jameslahm/RepViT}。