Major winning Convolutional Neural Networks (CNNs), such as AlexNet, VGGNet, ResNet, GoogleNet, include tens to hundreds of millions of parameters, which impose considerable computation and memory overhead. This limits their practical use for training, optimization and memory efficiency. On the contrary, light-weight architectures, being proposed to address this issue, mainly suffer from low accuracy. These inefficiencies mostly stem from following an ad hoc procedure. We propose a simple architecture, called SimpleNet, based on a set of designing principles, with which we empirically show, a well-crafted yet simple and reasonably deep architecture can perform on par with deeper and more complex architectures. SimpleNet provides a good tradeoff between the computation/memory efficiency and the accuracy. Our simple 13-layer architecture outperforms most of the deeper and complex architectures to date such as VGGNet, ResNet, and GoogleNet on several well-known benchmarks while having 2 to 25 times fewer number of parameters and operations. This makes it very handy for embedded systems or systems with computational and memory limitations. We achieved state-of-the-art result on CIFAR10 outperforming several heavier architectures, near state of the art on MNIST and competitive results on CIFAR100 and SVHN. We also outperformed the much larger and deeper architectures such as VGGNet and popular variants of ResNets among others on the ImageNet dataset. Models are made available at: https://github.com/Coderx7/SimpleNet

翻译：主流获胜的卷积神经网络（CNN），如AlexNet、VGGNet、ResNet、GoogleNet，包含数千万至数亿个参数，这带来了巨大的计算和内存开销，限制了它们在训练、优化和内存效率方面的实际应用。相反，为解决此问题而提出的轻量级架构主要面临精度低的问题。这些低效性大多源于采用临时性流程。我们提出了一种名为SimpleNet的简单架构，该架构基于一组设计原则。通过实验证明，一个精心设计、结构简单且深度适中的架构能够与更深层更复杂的架构性能相当。SimpleNet在计算/内存效率与精度之间实现了良好平衡。在多个知名基准测试中，我们简单的13层架构超越了现有大多数更深层复杂架构（如VGGNet、ResNet、GoogleNet），同时参数数量和运算量减少了2至25倍，这使得它非常适合嵌入式系统或受计算内存限制的系统。我们在CIFAR10上取得了超越多个重型架构的最先进结果，在MNIST上接近最先进水平，在CIFAR100和SVHN上取得了有竞争力的结果。此外，在ImageNet数据集上，我们也超越了包括VGGNet和ResNet主流变体在内的许多更大更深层架构。模型已开源在：https://github.com/Coderx7/SimpleNet