Population-based search has recently emerged as a possible alternative to Reinforcement Learning (RL) for black-box neural architecture search (NAS). It performs well in practice even though it is not theoretically well understood. In particular, whereas traditional population-based search methods such as evolutionary algorithms (EAs) draw much power from crossover operations, it is difficult to take advantage of them in NAS. The main obstacle is believed to be the permutation problem: The mapping between genotype and phenotype in traditional graph representations is many-to-one, leading to a disruptive effect of standard crossover. This paper presents the first theoretical analysis of the behaviors of mutation, crossover and RL in black-box NAS, and proposes a new crossover operator based on the shortest edit path (SEP) in graph space. The SEP crossover is shown theoretically to overcome the permutation problem, and as a result, have a better expected improvement compared to mutation, standard crossover and RL. Further, it empirically outperform these other methods on state-of-the-art NAS benchmarks. The SEP crossover therefore allows taking full advantage of population-based search in NAS, and the underlying theory can serve as a foundation for deeper understanding of black-box NAS methods in general.

翻译：基于种群的搜索最近被提出作为黑箱神经架构搜索（NAS）中强化学习（RL）的一种可能替代方案。尽管其在理论上尚未被充分理解，但在实践中表现良好。特别是，虽然传统的基于种群的搜索方法（如进化算法（EA））从交叉操作中汲取了大量能力，但在NAS中利用这些操作却很困难。主要障碍被认为是排列问题：传统图表示中基因型与表型之间的映射是多对一的，导致标准交叉产生破坏性效应。本文首次对黑箱NAS中变异、交叉和RL的行为进行了理论分析，并基于图空间中的最短编辑路径（SEP）提出了一种新的交叉算子。理论上证明，SEP交叉能够克服排列问题，因此与变异、标准交叉和RL相比，具有更好的期望改进。此外，它在最先进的NAS基准测试中经验性地优于这些其他方法。因此，SEP交叉使得在NAS中充分利用基于种群的搜索成为可能，且其底层理论可作为更深入理解一般黑箱NAS方法的基础。