The deployment of Deep Neural Networks (DNNs) on edge devices is hindered by the substantial gap between performance requirements and available processing power. While recent research has made significant strides in developing pruning methods to build a sparse network for reducing the computing overhead of DNNs, there remains considerable accuracy loss, especially at high pruning ratios. We find that the architectures designed for dense networks by differentiable architecture search methods are ineffective when pruning mechanisms are applied to them. The main reason is that the current method does not support sparse architectures in their search space and uses a search objective that is made for dense networks and does not pay any attention to sparsity. In this paper, we propose a new method to search for sparsity-friendly neural architectures. We do this by adding two new sparse operations to the search space and modifying the search objective. We propose two novel parametric SparseConv and SparseLinear operations in order to expand the search space to include sparse operations. In particular, these operations make a flexible search space due to using sparse parametric versions of linear and convolution operations. The proposed search objective lets us train the architecture based on the sparsity of the search space operations. Quantitative analyses demonstrate that our search architectures outperform those used in the stateof-the-art sparse networks on the CIFAR-10 and ImageNet datasets. In terms of performance and hardware effectiveness, DASS increases the accuracy of the sparse version of MobileNet-v2 from 73.44% to 81.35% (+7.91% improvement) with 3.87x faster inference time.

翻译：深度神经网络在边缘设备上的部署受到性能需求与可用算力之间巨大差距的制约。尽管近期研究在开发剪枝方法以构建稀疏网络、降低深度神经网络计算开销方面取得了显著进展，但在高剪枝率下仍存在显著的精度损失。我们发现，当对密集网络应用可微分架构搜索方法所设计的架构施加剪枝机制时，这些架构效果不佳。主要原因在于现有方法在搜索空间中不支持稀疏架构，且采用的搜索目标专为密集网络设计，未考虑稀疏性。本文提出了一种搜索稀疏友好型神经架构的新方法。我们通过向搜索空间添加两种新的稀疏操作并修改搜索目标来实现这一目标。我们提出了两种新型参数化稀疏操作SparseConv和SparseLinear，以扩展搜索空间包容稀疏操作。具体而言，这些操作通过使用线性操作和卷积操作的稀疏参数化版本，构建了灵活的搜索空间。所提出的搜索目标使我们能够基于搜索空间操作的稀疏性训练架构。定量分析表明，我们的搜索架构在CIFAR-10和ImageNet数据集上优于当前最优稀疏网络所使用的架构。在性能和硬件效率方面，DASS将稀疏版MobileNet-v2的准确率从73.44%提升至81.35%（提升7.91%），同时推理速度加快3.87倍。