Various hand-designed CNN architectures have been developed, such as VGG, ResNet, DenseNet, etc., and achieve State-of-the-Art (SoTA) levels on different tasks. Neural Architecture Search (NAS) now focuses on automatically finding the best CNN architecture to handle the above tasks. However, the verification of a searched architecture is very time-consuming and makes predictor-based methods become an essential and important branch of NAS. Two commonly used techniques to build predictors are graph-convolution networks (GCN) and multilayer perceptron (MLP). In this paper, we consider the difference between GCN and MLP on adjacent operation trails and then propose the Redirected Adjacent Trails NAS (RATs-NAS) to quickly search for the desired neural network architecture. The RATs-NAS consists of two components: the Redirected Adjacent Trails GCN (RATs-GCN) and the Predictor-based Search Space Sampling (P3S) module. RATs-GCN can change trails and their strengths to search for a better neural network architecture. DSS can rapidly focus on tighter intervals of FLOPs in the search space. Based on our observations on cell-based NAS, we believe that architectures with similar FLOPs will perform similarly. Finally, the RATs-NAS consisting of RATs-GCN and DSS beats WeakNAS, Arch-Graph, and others by a significant margin on three sub-datasets of NASBench-201.
翻译:各类手工设计的CNN架构(如VGG、ResNet、DenseNet等)已在不同任务上达到最优水平。神经架构搜索(NAS)目前聚焦于自动寻找最佳CNN架构以处理上述任务。然而,搜索所得架构的验证过程极为耗时,这使得基于预测器的方法成为NAS中不可或缺的重要分支。构建预测器的两种常用技术是图卷积网络(GCN)和多层感知机(MLP)。本文从相邻操作路径的差异入手,提出重定向相邻路径NAS(RATs-NAS)以快速搜索所需神经网络架构。RATs-NAS包含两个组件:重定向相邻路径GCN(RATs-GCN)和基于预测器的搜索空间采样(P3S)模块。RATs-GCN能够改变路径及其强度以搜索更优的神经网络架构,而DSS可快速聚焦搜索空间中FLOPs的紧致区间。基于我们对细胞级NAS的观察,我们认为具有相似FLOPs的架构会表现相近。最终,由RATs-GCN与DSS构成的RATs-NAS在NASBench-201的三个子数据集上显著超越了WeakNAS、Arch-Graph等方法。