Population-based search has recently emerged as a possible alternative to Reinforcement Learning (RL) for black-box neural architecture search (NAS). It performs well in practice even though it is not theoretically well understood. In particular, whereas traditional population-based search methods such as evolutionary algorithms (EAs) draw much power from crossover operations, it is difficult to take advantage of them in NAS. The main obstacle is believed to be the permutation problem: The mapping between genotype and phenotype in traditional graph representations is many-to-one, leading to a disruptive effect of standard crossover. This paper presents the first theoretical analysis of the behaviors of mutation, crossover and RL in black-box NAS, and proposes a new crossover operator based on the shortest edit path (SEP) in graph space. The SEP crossover is shown theoretically to overcome the permutation problem, and as a result, have a better expected improvement compared to mutation, standard crossover and RL. Further, it empirically outperform these other methods on state-of-the-art NAS benchmarks. The SEP crossover therefore allows taking full advantage of population-based search in NAS, and the underlying theory can serve as a foundation for deeper understanding of black-box NAS methods in general.

翻译：基于种群的搜索方法近期被提出作为黑箱神经架构搜索（NAS）中强化学习（RL）的潜在替代方案。尽管其理论尚未被充分理解，但在实践中表现出色。具体而言，虽然演化算法（EA）等传统种群搜索方法从交叉操作中汲取了大量能力，但在NAS中难以充分利用这些优势。主要障碍被认为是置换问题：传统图表示中基因型与表现型之间的映射是多对一的，导致标准交叉产生破坏性效果。本文首次对黑箱NAS中变异、交叉和RL的行为进行了理论分析，并提出了一种基于图空间中最短编辑路径（SEP）的新型交叉算子。理论上证明了SEP交叉能够克服置换问题，因此相较于变异、标准交叉和RL具有更优的期望改进。此外，在先进的NAS基准测试中，该算子性能实证优于其他方法。因此，SEP交叉允许在NAS中充分利用基于种群的搜索，其基础理论可作为深入理解一般黑箱NAS方法的基石。