We consider a simple setting in neuroevolution where an evolutionary algorithm optimizes the weights and activation functions of a simple artificial neural network. We then define simple example functions to be learned by the network and conduct rigorous runtime analyses for networks with a single neuron and for a more advanced structure with several neurons and two layers. Our results show that the proposed algorithm is generally efficient on two example problems designed for one neuron and efficient with at least constant probability on the example problem for a two-layer network. In particular, the so-called harmonic mutation operator choosing steps of size $j$ with probability proportional to $1/j$ turns out as a good choice for the underlying search space. However, for the case of one neuron, we also identify situations with hard-to-overcome local optima. Experimental investigations of our neuroevolutionary algorithm and a state-of-the-art CMA-ES support the theoretical findings.

翻译：我们考虑神经进化中的一个简单场景：进化算法优化简单人工神经网络的权重和激活函数。接着，我们定义需要网络学习的简单示例函数，并对单神经元网络以及包含多个神经元和两层的更复杂结构进行严格运行时分析。结果表明，所提算法在针对单神经元设计的两个示例问题上总体高效，在针对双层网络的示例问题上至少能以恒定概率高效运行。特别地，步长 $j$ 的选择概率与 $1/j$ 成正比的所谓调和变异算子，在底层搜索空间中表现出良好的适应性。然而，在单神经元情形下，我们同样识别出存在难以克服的局部最优解的情况。对我们的神经进化算法及先进CMA-ES的实验研究支持了理论发现。