We present an efficient alternative to the convolutional layer through utilising spatial basis filters (SBF). SBF layers exploit the spatial redundancy in the convolutional filters across the depth to achieve overall model compression, while maintaining the top-end accuracy of their dense counter-parts. Training SBF-Nets is modelled as a simple pruning problem, but instead of zeroing out the pruned channels, they are replaced with inexpensive transformations from the set of non-pruned features. To enable an adoption of these SBF layers, we provide a flexible training pipeline and an efficient implementation in CUDA with low latency. To further demonstrate the effective capacity of these models, we apply semi-supervised knowledge distillation that leads to significant performance improvements over the baseline networks. Our experiments show that SBF-Nets are effective and achieve comparable or improved performance to state-of-the-art across CIFAR10, CIFAR100, Tiny-ImageNet, and ILSCRC-2012.

翻译：我们通过使用空间基础过滤器(SBF),为革命层提供了一种高效的替代方法。SBF层利用革命层过滤器的深度空间冗余,以实现总体模型压缩,同时保持其密度高的对子部分的顶端精确度。培训SBF-Net的模型是一个简单的修补问题,但是,它不是从小管线上排出,而是用一套非修补功能的廉价转换来取代。为了能够采用SBF层,我们提供了灵活的培训管道,并在CUDA中以低延迟度高效实施。为了进一步展示这些模型的有效能力,我们采用了半受监督的知识蒸馏,从而大大改进了基线网络的性能。我们的实验表明,SBF-Net是有效的,在CIFAR10、CIFAR100、Tiny-ImageNet和ILSCRC-2012年之间实现了与最新技术的可比或改进性能。