The deployment of Deep Neural Networks (DNNs) on edge devices is hindered by the substantial gap between performance requirements and available processing power. While recent research has made significant strides in developing pruning methods to build a sparse network for reducing the computing overhead of DNNs, there remains considerable accuracy loss, especially at high pruning ratios. We find that the architectures designed for dense networks by differentiable architecture search methods are ineffective when pruning mechanisms are applied to them. The main reason is that the current method does not support sparse architectures in their search space and uses a search objective that is made for dense networks and does not pay any attention to sparsity. In this paper, we propose a new method to search for sparsity-friendly neural architectures. We do this by adding two new sparse operations to the search space and modifying the search objective. We propose two novel parametric SparseConv and SparseLinear operations in order to expand the search space to include sparse operations. In particular, these operations make a flexible search space due to using sparse parametric versions of linear and convolution operations. The proposed search objective lets us train the architecture based on the sparsity of the search space operations. Quantitative analyses demonstrate that our search architectures outperform those used in the stateof-the-art sparse networks on the CIFAR-10 and ImageNet datasets. In terms of performance and hardware effectiveness, DASS increases the accuracy of the sparse version of MobileNet-v2 from 73.44% to 81.35% (+7.91% improvement) with 3.87x faster inference time.

翻译：深度神经网络在边缘设备上的部署受到性能需求与可用算力之间巨大差距的制约。尽管近年研究通过开发剪枝方法构建稀疏网络以降低深度神经网络计算开销方面取得了显著进展，但在高剪枝率下仍存在显著的精度损失。我们发现，为密集网络设计的可微架构搜索方法在应用剪枝机制时效果不佳。其主要原因在于现有方法在搜索空间中不支持稀疏架构，且采用的搜索目标专为密集网络设计而未考虑稀疏性。本文提出一种搜索稀疏友好神经架构的新方法，通过向搜索空间添加两种新算子并修改搜索目标实现。我们提出两种新型参数化稀疏卷积和稀疏线性算子，使搜索空间扩展至包含稀疏操作。特别地，这些算子通过使用线性与卷积运算的稀疏参数化版本构建出灵活的搜索空间。所提出的搜索目标使我们能够基于搜索空间操作的稀疏性训练架构。定量分析表明，我们的搜索架构在CIFAR-10和ImageNet数据集上的表现优于当前最先进的稀疏网络。在性能和硬件效率方面，DASS使MobileNet-v2的稀疏版本精度从73.44%提升至81.35%（提升7.91%），同时推理速度加快3.87倍。