Network spaces have been known as a critical factor in both handcrafted network designs or defining search spaces for Neural Architecture Search (NAS). However, an effective space involves tremendous prior knowledge and/or manual effort, and additional constraints are required to discover efficiency-aware architectures. In this paper, we define a new problem, Network Space Search (NSS), as searching for favorable network spaces instead of a single architecture. We propose an NSS method to directly search for efficient-aware network spaces automatically, reducing the manual effort and immense cost in discovering satisfactory ones. The resultant network spaces, named Elite Spaces, are discovered from Expanded Search Space with minimal human expertise imposed. The Pareto-efficient Elite Spaces are aligned with the Pareto front under various complexity constraints and can be further served as NAS search spaces, benefiting differentiable NAS approaches (e.g. In CIFAR-100, an averagely 2.3% lower error rate and 3.7% closer to target constraint than the baseline with around 90% fewer samples required to find satisfactory networks). Moreover, our NSS approach is capable of searching for superior spaces in future unexplored spaces, revealing great potential in searching for network spaces automatically.

翻译：在人工设计的网络设计或界定神经结构搜索搜索空间(NAS)中,网络空间被认为是一个关键因素。然而,有效的空间需要大量的事先知识和(或)人工努力,而发现高效率建筑需要额外的限制。在本文中,我们定义了一个新的问题,即网络空间搜索(NSS),以寻找有利的网络空间,而不是单一的建筑。我们提议了一种NSS方法,以自动直接搜索高效率的观测网络空间,减少人工操作和发现满意的网络空间的巨额费用。由此产生的称为Elite空间的网络空间,是在人类专门知识最少的扩大搜索空间中发现的。Pareto高效的 Elite空间与Pareto前方的复杂限制一致,可以进一步作为NAS搜索空间,有利于不同的NAS方法(例如,在CIFAR-100中,平均低2.3%的误差率和接近目标制约的3.7%,比基线要低约90%的样本,以找到令人满意的网络)。此外,我们的NSS方法能够在未来未探索的空间中自动搜索高级空间,揭示巨大的潜力。