This survey focuses on categorizing and evaluating the methods of supernet optimization in the field of Neural Architecture Search (NAS). Supernet optimization involves training a single, over-parameterized network that encompasses the search space of all possible network architectures. The survey analyses supernet optimization methods based on their approaches to spatial and temporal optimization. Spatial optimization relates to optimizing the architecture and parameters of the supernet and its subnets, while temporal optimization deals with improving the efficiency of selecting architectures from the supernet. The benefits, limitations, and potential applications of these methods in various tasks and settings, including transferability, domain generalization, and Transformer models, are also discussed.

翻译：本综述聚焦于神经架构搜索（NAS）领域中超网络优化方法的分类与评估。超网络优化涉及训练一个单一且过参数化的网络，该网络囊括所有可能网络架构的搜索空间。本研究基于空间优化与时间优化两种方法对超网络优化技术进行分析。空间优化涉及超网络及其子网的架构与参数优化，而时间优化则关注提升从超网络中选取架构的效率。此外，本文还讨论了这些方法在各类任务与场景（包括迁移性、领域泛化以及Transformer模型）中的优势、局限及潜在应用。