Recently, zero-shot (or training-free) Neural Architecture Search (NAS) approaches have been proposed to liberate NAS from the expensive training process. The key idea behind zero-shot NAS approaches is to design proxies that can predict the accuracy of some given networks without training the network parameters. The proxies proposed so far are usually inspired by recent progress in theoretical understanding of deep learning and have shown great potential on several datasets and NAS benchmarks. This paper aims to comprehensively review and compare the state-of-the-art (SOTA) zero-shot NAS approaches, with an emphasis on their hardware awareness. To this end, we first review the mainstream zero-shot proxies and discuss their theoretical underpinnings. We then compare these zero-shot proxies through large-scale experiments and demonstrate their effectiveness in both hardware-aware and hardware-oblivious NAS scenarios. Finally, we point out several promising ideas to design better proxies. Our source code and the list of related papers are available on https://github.com/SLDGroup/survey-zero-shot-nas.

翻译：近年来，零样本（或无训练）神经架构搜索方法被提出，旨在将NAS从昂贵的训练过程中解放出来。零样本NAS方法的核心思想是设计无需训练网络参数即可预测给定网络性能的代理指标。现有代理指标通常受到深度学习理论最新进展的启发，并在多个数据集和NAS基准测试中展现出巨大潜力。本文旨在全面综述和比较最先进的零样本NAS方法，重点关注其硬件感知特性。为此，我们首先回顾主流零样本代理指标并探讨其理论基础；随后通过大规模实验比较这些代理指标，验证其在硬件感知与硬件无关NAS场景中的有效性；最后提出若干设计更优代理指标的潜在研究方向。相关源代码及论文列表已发布于https://github.com/SLDGroup/survey-zero-shot-nas。