Neural Architecture Search (NAS), aiming at automatically designing neural architectures by machines, has been considered a key step toward automatic machine learning. One notable NAS branch is the weight-sharing NAS, which significantly improves search efficiency and allows NAS algorithms to run on ordinary computers. Despite receiving high expectations, this category of methods suffers from low search effectiveness. By employing a generalization boundedness tool, we demonstrate that the devil behind this drawback is the untrustworthy architecture rating with the oversized search space of the possible architectures. Addressing this problem, we modularize a large search space into blocks with small search spaces and develop a family of models with the distilling neural architecture (DNA) techniques. These proposed models, namely a DNA family, are capable of resolving multiple dilemmas of the weight-sharing NAS, such as scalability, efficiency, and multi-modal compatibility. Our proposed DNA models can rate all architecture candidates, as opposed to previous works that can only access a sub- search space using heuristic algorithms. Moreover, under a certain computational complexity constraint, our method can seek architectures with different depths and widths. Extensive experimental evaluations show that our models achieve state-of-the-art top-1 accuracy of 78.9% and 83.6% on ImageNet for a mobile convolutional network and a small vision transformer, respectively. Additionally, we provide in-depth empirical analysis and insights into neural architecture ratings. Codes available: \url{https://github.com/changlin31/DNA}.

翻译：神经架构搜索（NAS）旨在通过机器自动设计神经架构，已被视为实现自动机器学习的关键步骤。其中，权重共享NAS是一个重要分支，它显著提升了搜索效率，使得NAS算法能够在普通计算机上运行。尽管这类方法备受期待，但其搜索有效性较低。通过采用泛化有界性工具，我们证明这一缺陷背后的原因是：在过大的架构搜索空间中，架构评级不可靠。为解决该问题，我们将大搜索空间模块化为多个具有小搜索空间的块，并利用蒸馏神经架构（DNA）技术开发了一系列模型。这些被提出的模型，即DNA家族，能够解决权重共享NAS面临的多个困境，如可扩展性、效率和多模态兼容性。与以往仅能通过启发式算法访问子搜索空间的工作不同，我们提出的DNA模型能够对所有架构候选项进行评级。此外，在特定的计算复杂度约束下，我们的方法可以搜索具有不同深度和宽度的架构。大量实验评估表明，我们的模型在ImageNet数据集上，针对移动卷积网络和小型视觉Transformer分别实现了78.9%和83.6%的顶尖Top-1准确率。此外，我们提供了关于神经架构评级的深入实证分析和见解。代码地址：\url{https://github.com/changlin31/DNA}。