Recently, zero-shot (or training-free) Neural Architecture Search (NAS) approaches have been proposed to liberate the NAS from training requirements. The key idea behind zero-shot NAS approaches is to design proxies that predict the accuracies of the given networks without training network parameters. The proxies proposed so far are usually inspired by recent progress in theoretical deep learning and have shown great potential on several NAS benchmark datasets. This paper aims to comprehensively review and compare the state-of-the-art (SOTA) zero-shot NAS approaches, with an emphasis on their hardware awareness. To this end, we first review the mainstream zero-shot proxies and discuss their theoretical underpinnings. We then compare these zero-shot proxies through large-scale experiments and demonstrate their effectiveness in both hardware-aware and hardware-oblivious NAS scenarios. Finally, we point out several promising ideas to design better proxies. Our source code and the related paper list are available on https://github.com/SLDGroup/survey-zero-shot-nas.

翻译：近期，零样本（即免训练）神经架构搜索（NAS）方法被提出，旨在使NAS摆脱对训练的需求。零样本NAS方法的核心思想是设计代理指标，无需训练网络参数即可预测给定网络的精度。目前提出的代理指标通常受到理论深度学习最新进展的启发，并在多个NAS基准数据集上展现出巨大潜力。本文旨在全面综述和比较最先进的零样本NAS方法，并重点关注其硬件感知能力。为此，我们首先回顾主流零样本代理指标，并探讨其理论基础。随后，通过大规模实验对比这些零样本代理指标，证明其在硬件感知与硬件无关的NAS场景中的有效性。最后，我们指出若干具有前景的设计更优代理指标的思路。我们的源代码及相关论文列表已公开于 https://github.com/SLDGroup/survey-zero-shot-nas。