We consider a simple setting in neuroevolution where an evolutionary algorithm optimizes the weights and activation functions of a simple artificial neural network. We then define simple example functions to be learned by the network and conduct rigorous runtime analyses for networks with a single neuron and for a more advanced structure with several neurons and two layers. Our results show that the proposed algorithm is generally efficient on two example problems designed for one neuron and efficient with at least constant probability on the example problem for a two-layer network. In particular, the so-called harmonic mutation operator choosing steps of size $j$ with probability proportional to $1/j$ turns out as a good choice for the underlying search space. However, for the case of one neuron, we also identify situations with hard-to-overcome local optima. Experimental investigations of our neuroevolutionary algorithm and a state-of-the-art CMA-ES support the theoretical findings.

翻译：本文考虑神经进化中的简单设置，其中进化算法优化简单人工神经网络的权重和激活函数。我们定义了需要网络学习的简单示例函数，并对单神经元网络以及具有多个神经元和两层结构的更复杂网络进行了严格的运行时分析。结果表明，所提出的算法在针对单神经元设计的两个示例问题上总体高效，在针对两层网络的示例问题上至少以恒定概率高效。特别地，选择步长$j$的概率与$1/j$成正比的所谓调和变异算子被证明是底层搜索空间中的良好选择。然而，对于单神经元的情况，我们也识别出存在难以克服的局部最优解的情况。对我们的神经进化算法和最新CMA-ES的实验研究支持了理论发现。