Recently, lightweight Vision Transformers (ViTs) demonstrate superior performance and lower latency compared with lightweight Convolutional Neural Networks (CNNs) on resource-constrained mobile devices. This improvement is usually attributed to the multi-head self-attention module, which enables the model to learn global representations. However, the architectural disparities between lightweight ViTs and lightweight CNNs have not been adequately examined. In this study, we revisit the efficient design of lightweight CNNs and emphasize their potential for mobile devices. We incrementally enhance the mobile-friendliness of a standard lightweight CNN, specifically MobileNetV3, by integrating the efficient architectural choices of lightweight ViTs. This ends up with a new family of pure lightweight CNNs, namely RepViT. Extensive experiments show that RepViT outperforms existing state-of-the-art lightweight ViTs and exhibits favorable latency in various vision tasks. On ImageNet, RepViT achieves over 80\% top-1 accuracy with 1ms latency on an iPhone 12, which is the first time for a lightweight model, to the best of our knowledge. Our largest model, RepViT-M2.3, obtains 83.7\% accuracy with only 2.3ms latency. The code and trained models are available at \url{https://github.com/jameslahm/RepViT}.

翻译：近期，轻量级视觉Transformer（ViT）在资源受限的移动设备上展现出比轻量级卷积神经网络（CNN）更优的性能和更低的延迟。这一提升通常归因于多头自注意力模块，该模块使模型能够学习全局表征。然而，轻量级ViT与轻量级CNN之间的架构差异尚未得到充分探究。在本研究中，我们重新审视了轻量级CNN的高效设计，并强调了其在移动设备上的潜力。我们通过整合轻量级ViT的高效架构选择，逐步增强标准轻量级CNN（具体为MobileNetV3）的移动端友好性，最终衍生出一系列全新的纯轻量级CNN——RepViT。大量实验表明，RepViT在多种视觉任务中超越现有最先进的轻量级ViT，并展现出理想的延迟表现。在ImageNet上，RepViT在iPhone 12上以1ms延迟实现超过80%的Top-1准确率——据我们所知，这是轻量级模型首次达成此成绩。我们的最大模型RepViT-M2.3在仅2.3ms延迟下获得83.7%的准确率。代码与预训练模型已开源至 \url{https://github.com/jameslahm/RepViT}。