The wide application of pre-trained models is driving the trend of once-for-all training in one-shot neural architecture search (NAS). However, training within a huge sample space damages the performance of individual subnets and requires much computation to search for an optimal model. In this paper, we present PreNAS, a search-free NAS approach that accentuates target models in one-shot training. Specifically, the sample space is dramatically reduced in advance by a zero-cost selector, and weight-sharing one-shot training is performed on the preferred architectures to alleviate update conflicts. Extensive experiments have demonstrated that PreNAS consistently outperforms state-of-the-art one-shot NAS competitors for both Vision Transformer and convolutional architectures, and importantly, enables instant specialization with zero search cost. The code will be released upon acceptance.

翻译：预训练模型的广泛应用正推动着一次性神经架构搜索（NAS）中“一次训练，适用于所有架构”的趋势。然而，在庞大的样本空间中进行训练会损害单个子网络的性能，并且需要大量计算来搜索最优模型。本文提出PreNAS，一种无搜索的NAS方法，该方法在一次训练中突出目标模型。具体而言，通过零成本选择器预先大幅缩减样本空间，并在偏好架构上执行权重共享的一次性训练以缓解更新冲突。大量实验证明，PreNAS在视觉Transformer和卷积架构上均持续优于最先进的一次性NAS竞争对手，且重要的是，能够以零搜索成本实现即时定制。代码将在论文接收后发布。