In this work, we show that simultaneously training and mixing neural networks is a promising way to conduct Neural Architecture Search (NAS). For hyperparameter optimization, reusing the partially trained weights allows for efficient search, as was previously demonstrated by the Population Based Training (PBT) algorithm. We propose PBT-NAS, an adaptation of PBT to NAS where architectures are improved during training by replacing poorly-performing networks in a population with the result of mixing well-performing ones and inheriting the weights using the shrink-perturb technique. After PBT-NAS terminates, the created networks can be directly used without retraining. PBT-NAS is highly parallelizable and effective: on challenging tasks (image generation and reinforcement learning) PBT-NAS achieves superior performance compared to baselines (random search and mutation-based PBT).

翻译：本文表明，同时训练和混合神经网络是进行神经架构搜索（NAS）的一种有效途径。在超参数优化中，复用部分训练权重可支持高效搜索，这一点已由种群训练（PBT）算法先前证明。我们提出PBT-NAS，即PBT在NAS中的改进方案：在训练过程中，通过将种群中性能较差的网络替换为混合性能优异网络的结果，并利用shrink-perturb技术继承其权重，从而优化架构。PBT-NAS终止后，生成的网络可直接使用，无需重新训练。该方法具有高度并行性和有效性：在图像生成和强化学习等挑战性任务中，PBT-NAS相比基线方法（随机搜索和基于突变的PBT）取得了更优性能。