We formalize and analyze a fundamental component of differentiable neural architecture search (NAS): local "operation scoring" at each operation choice. We view existing operation scoring functions as inexact proxies for accuracy, and we find that they perform poorly when analyzed empirically on NAS benchmarks. From this perspective, we introduce a novel \textit{perturbation-based zero-cost operation scoring} (Zero-Cost-PT) approach, which utilizes zero-cost proxies that were recently studied in multi-trial NAS but degrade significantly on larger search spaces, typical for differentiable NAS. We conduct a thorough empirical evaluation on a number of NAS benchmarks and large search spaces, from NAS-Bench-201, NAS-Bench-1Shot1, NAS-Bench-Macro, to DARTS-like and MobileNet-like spaces, showing significant improvements in both search time and accuracy. On the ImageNet classification task on the DARTS search space, our approach improved accuracy compared to the best current training-free methods (TE-NAS) while being over 10$\times$ faster (total searching time 25 minutes on a single GPU), and observed significantly better transferability on architectures searched on the CIFAR-10 dataset with an accuracy increase of 1.8 pp. Our code is available at: https://github.com/zerocostptnas/zerocost_operation_score.

翻译：我们形式化并分析了可微分神经架构搜索（NAS）的一个基本组件：每个操作选择处的局部"操作评分"。我们将现有操作评分函数视为准确率的非精确代理，并发现它们在NAS基准上进行实证分析时表现不佳。基于这一视角，我们提出了一种新颖的基于扰动的零代价操作评分方法（Zero-Cost-PT），该方法利用了最近在多试验NAS中研究但在典型可微分NAS的更大搜索空间上性能显著下降的零代价代理。我们在多个NAS基准和大型搜索空间上进行了彻底的实证评估，涵盖NAS-Bench-201、NAS-Bench-1Shot1、NAS-Bench-Macro、DARTS类空间和MobileNet类空间，结果显示搜索时间和准确率均显著提升。在DARTS搜索空间的ImageNet分类任务上，与当前最优的无训练方法（TE-NAS）相比，我们的方法在提高准确率的同时速度提升超过10倍（单GPU总搜索时间25分钟），并且在CIFAR-10数据集上搜索到的架构上观察到显著更好的迁移性，准确率提升1.8个百分点。我们的代码开源地址：https://github.com/zerocostptnas/zerocost_operation_score。