To design fast neural networks, many works have been focusing on reducing the number of floating-point operations (FLOPs). We observe that such reduction in FLOPs, however, does not necessarily lead to a similar level of reduction in latency. This mainly stems from inefficiently low floating-point operations per second (FLOPS). To achieve faster networks, we revisit popular operators and demonstrate that such low FLOPS is mainly due to frequent memory access of the operators, especially the depthwise convolution. We hence propose a novel partial convolution (PConv) that extracts spatial features more efficiently, by cutting down redundant computation and memory access simultaneously. Building upon our PConv, we further propose FasterNet, a new family of neural networks, which attains substantially higher running speed than others on a wide range of devices, without compromising on accuracy for various vision tasks. For example, on ImageNet-1k, our tiny FasterNet-T0 is $2.8\times$, $3.3\times$, and $2.4\times$ faster than MobileViT-XXS on GPU, CPU, and ARM processors, respectively, while being $2.9\%$ more accurate. Our large FasterNet-L achieves impressive $83.5\%$ top-1 accuracy, on par with the emerging Swin-B, while having $36\%$ higher inference throughput on GPU, as well as saving $37\%$ compute time on CPU. Code is available at \url{https://github.com/JierunChen/FasterNet}.

翻译：为设计快速神经网络，许多研究致力于减少浮点运算次数（FLOPs）。然而，我们观察到FLOPs的减少并不必然带来同等程度的延迟降低，这主要源于低效的每秒浮点运算次数（FLOPS）。为实现更快速的网络，我们重新审视常用算子，并论证此类低FLOPS主要源于算子的频繁内存访问，尤其是深度可分离卷积。为此，我们提出新型部分卷积（PConv），通过同时削减冗余计算与内存访问，更高效地提取空间特征。基于PConv，我们进一步提出新一代神经网络家族FasterNet，其在多种设备上均显著超越其他网络的运行速度，且不损害各类视觉任务的精度。例如，在ImageNet-1k上，微型FasterNet-T0在GPU、CPU及ARM处理器上分别比MobileViT-XXS快2.8倍、3.3倍和2.4倍，同时精度提升2.9%。大型FasterNet-L达到83.5%的顶尖Top-1准确率，与新兴的Swin-B相当，而GPU推理吞吐量高出36%，CPU计算时间节省37%。代码已开源至\url{https://github.com/JierunChen/FasterNet}。