Improving search efficiency serves as one of the crucial objectives of Neural Architecture Search (NAS). However, many current approaches ignore the universality of the search strategy and fail to reduce the computational redundancy during the search process, especially in one-shot NAS architectures. Besides, current NAS methods show invalid reparameterization in non-linear search space, leading to poor efficiency in common search spaces like DARTS. In this paper, we propose TopoNAS, a model-agnostic approach for gradient-based one-shot NAS that significantly reduces searching time and memory usage by topological simplification of searchable paths. Firstly, we model the non-linearity in search spaces to reveal the parameterization difficulties. To improve the search efficiency, we present a topological simplification method and iteratively apply module-sharing strategies to simplify the topological structure of searchable paths. In addition, a kernel normalization technique is also proposed to preserve the search accuracy. Experimental results on the NASBench201 benchmark with various search spaces demonstrate the effectiveness of our method. It proves the proposed TopoNAS enhances the performance of various architectures in terms of search efficiency while maintaining a high level of accuracy. The project page is available at https://xdedss.github.io/topo_simplification.

翻译：提升搜索效率是神经架构搜索（NAS）的关键目标之一。然而，当前许多方法忽视了搜索策略的普适性，未能有效减少搜索过程中的计算冗余，尤其是在一次性NAS架构中。此外，现有NAS方法在非线性搜索空间中表现出无效的重参数化，导致在DARTS等常见搜索空间中效率低下。本文提出TopoNAS，一种与模型无关的基于梯度一次性NAS方法，通过对可搜索路径进行拓扑简化，显著降低了搜索时间和内存占用。首先，我们对搜索空间中的非线性进行建模以揭示参数化难点。为提高搜索效率，我们提出一种拓扑简化方法，并迭代应用模块共享策略来简化可搜索路径的拓扑结构。此外，我们还提出核归一化技术以保持搜索精度。在NASBench201基准测试中针对多种搜索空间的实验结果表明了该方法的有效性。研究证明，所提出的TopoNAS在保持高精度的同时，显著提升了多种架构的搜索效率。项目页面详见https://xdedss.github.io/topo_simplification。