With the growing demand for vision applications and deployment across edge devices, the development of hardware-friendly architectures that maintain performance during device deployment becomes crucial. Neural architecture search (NAS) techniques explore various approaches to discover efficient architectures for diverse learning tasks in a computationally efficient manner. In this paper, we present the next-generation neural architecture design for computationally efficient neural architecture distillation - DONNAv2 . Conventional NAS algorithms rely on a computationally extensive stage where an accuracy predictor is learned to estimate model performance within search space. This building of accuracy predictors helps them predict the performance of models that are not being finetuned. Here, we have developed an elegant approach to eliminate building the accuracy predictor and extend DONNA to a computationally efficient setting. The loss metric of individual blocks forming the network serves as the surrogate performance measure for the sampled models in the NAS search stage. To validate the performance of DONNAv2 we have performed extensive experiments involving a range of diverse vision tasks including classification, object detection, image denoising, super-resolution, and panoptic perception network (YOLOP). The hardware-in-the-loop experiments were carried out using the Samsung Galaxy S10 mobile platform. Notably, DONNAv2 reduces the computational cost of DONNA by 10x for the larger datasets. Furthermore, to improve the quality of NAS search space, DONNAv2 leverages a block knowledge distillation filter to remove blocks with high inference costs.

翻译：随着视觉应用需求的增长及在边缘设备上的部署，开发兼顾硬件友好性且能保持部署性能的架构变得至关重要。神经架构搜索（NAS）技术探索多种方法，以计算高效的方式为不同学习任务发现高效架构。本文提出下一代计算高效神经架构蒸馏方法——DONNAv2。传统NAS算法依赖计算量庞大的阶段：学习一个精度预测器来估计搜索空间内模型的性能。这种构建精度预测器的方法有助于预测未微调模型的性能。我们在此开发了一种优雅的方法，无需构建精度预测器，即可将DONNA扩展至计算高效的场景。在NAS搜索阶段，构成网络的各独立模块的损失指标作为采样模型的替代性能度量。为验证DONNAv2的性能，我们开展了涉及多种视觉任务的广泛实验，包括分类、目标检测、图像去噪、超分辨率及全景感知网络（YOLOP）。采用三星Galaxy S10移动平台进行硬件在环实验。值得注意的是，DONNAv2将DONNA在较大数据集上的计算成本降低了10倍。此外，为提升NAS搜索空间质量，DONNAv2利用模块知识蒸馏过滤器，剔除推理成本高的模块。