Evolutionary neural architecture search (ENAS) employs evolutionary algorithms to find high-performing neural architectures automatically, and has achieved great success. However, compared to the empirical success, its rigorous theoretical analysis has yet to be touched. This work goes preliminary steps toward the mathematical runtime analysis of ENAS. In particular, we define a binary classification problem UNIFORM, and formulate an explicit fitness function to represent the relationship between neural architecture and classification accuracy. Furthermore, we consider (1+1)-ENAS algorithm with mutation to optimize the neural architecture, and obtain the following runtime bounds: 1) the one-bit mutation finds the optimum in an expected runtime of $O(n)$ and $\Omega(\log n)$; 2) the multi-bit mutation finds the optimum in an expected runtime of $\Theta(n)$. These theoretical results show that one-bit and multi-bit mutations achieve nearly the same performance on UNIFORM. We provide insight into the choices of mutation in the ENAS community: although multi-bit mutation can change the step size to prevent a local trap, this may not always improve runtime. Empirical results also verify the equivalence of these two mutation operators. This work begins the runtime analysis of ENAS, laying the foundation for further theoretical studies to guide the design of ENAS.

翻译：进化神经架构搜索（ENAS）采用进化算法自动寻找高性能神经架构，并已取得巨大成功。然而，与经验性成功相比，其严格的理论分析尚未触及。本研究初步探索了ENAS的数学运行时分析。具体而言，我们定义了一个二分类问题UNIFORM，并构建了一个显式适应度函数来表示神经架构与分类精度之间的关系。此外，我们考虑了带变异的(1+1)-ENAS算法来优化神经架构，并得到以下运行时边界：1) 单比特变异在期望运行时间$O(n)$和$\Omega(\log n)$内找到最优解；2) 多比特变异在期望运行时间$\Theta(n)$内找到最优解。这些理论结果表明，单比特与多比特变异在UNIFORM问题上性能几乎相同。我们为ENAS社区中变异的选择提供了见解：尽管多比特变异能改变步长以避免局部陷阱，但这并不总能提升运行时性能。实验结果也验证了这两种变异算子的等价性。本研究开启了ENAS的运行时分析，为后续指导ENAS设计的理论探索奠定了基础。