Search spaces hallmark the advancement of Neural Architecture Search (NAS). Large and complex search spaces with versatile building operators and structures provide more opportunities to brew promising architectures, yet pose severe challenges on efficient exploration and exploitation. Subsequently, several search space shrinkage methods optimize by selecting a single sub-region that contains some well-performing networks. Small performance and efficiency gains are observed with these methods but such techniques leave room for significantly improved search performance and are ineffective at retaining architectural diversity. We propose LISSNAS, an automated algorithm that shrinks a large space into a diverse, small search space with SOTA search performance. Our approach leverages locality, the relationship between structural and performance similarity, to efficiently extract many pockets of well-performing networks. We showcase our method on an array of search spaces spanning various sizes and datasets. We accentuate the effectiveness of our shrunk spaces when used in one-shot search by achieving the best Top-1 accuracy in two different search spaces. Our method achieves a SOTA Top-1 accuracy of 77.6\% in ImageNet under mobile constraints, best-in-class Kendal-Tau, architectural diversity, and search space size.

翻译：搜索空间标志着神经架构搜索（NAS）的进步。具有多功能构建算子和结构的大型复杂搜索空间为培育有前景的架构提供了更多机会，但也对高效探索与利用提出了严峻挑战。随后，一些搜索空间收缩方法通过选择包含若干表现优异网络的单一子区域进行优化。此类方法虽能观察到小幅性能与效率提升，但仍有显著改进搜索性能的空间，且难以保持架构多样性。我们提出LISSNAS，这是一种自动算法，可将大型搜索空间收缩为具有SOTA搜索性能的多样化小型搜索空间。我们的方法利用局部性——即结构与性能相似性之间的关系——高效提取多个性能优异的网络聚集区域。我们在涵盖不同规模与数据集的一系列搜索空间上展示了该方法。通过在一次搜索中实现两个不同搜索空间上的最佳Top-1准确率，我们突显了收缩空间的有效性。在移动约束条件下，我们的方法在ImageNet上达到77.6%的SOTA Top-1准确率，并在Kendal-Tau系数、架构多样性及搜索空间规模方面均达到最优水平。