Evolutionary Algorithms (EAs) play a crucial role in the architectural configuration and training of Artificial Deep Neural Networks (DNNs), a process known as neuroevolution. However, neuroevolution is hindered by its inherent computational expense, requiring multiple generations, a large population, and numerous epochs. The most computationally intensive aspect lies in evaluating the fitness function of a single candidate solution. To address this challenge, we employ Surrogate-assisted EAs (SAEAs). While a few SAEAs approaches have been proposed in neuroevolution, none have been applied to truly large DNNs due to issues like intractable information usage. In this work, drawing inspiration from Genetic Programming semantics, we use phenotypic distance vectors, outputted from DNNs, alongside Kriging Partial Least Squares (KPLS), an approach that is effective in handling these large vectors, making them suitable for search. Our proposed approach, named Neuro-Linear Genetic Programming surrogate model (NeuroLGP-SM), efficiently and accurately estimates DNN fitness without the need for complete evaluations. NeuroLGP-SM demonstrates competitive or superior results compared to 12 other methods, including NeuroLGP without SM, convolutional neural networks, support vector machines, and autoencoders. Additionally, it is worth noting that NeuroLGP-SM is 25% more energy-efficient than its NeuroLGP counterpart. This efficiency advantage adds to the overall appeal of our proposed NeuroLGP-SM in optimising the configuration of large DNNs.

翻译：摘要：进化算法在人工深度神经网络的架构配置与训练中扮演关键角色，这一过程被称为神经演化。然而，神经演化因其固有的计算开销而受到制约，需要多代迭代、大种群规模及大量训练周期。其中计算最密集的环节在于评估单个候选解的适应度函数。为解决该挑战，我们采用代理辅助进化算法。尽管已有少量代理辅助进化算法应用于神经演化领域，但由于存在信息利用困难等问题，这些方法均未真正应用于大型深度神经网络。本研究借鉴遗传编程语义学思想，使用深度神经网络输出的表型距离向量，结合能够有效处理大型向量的克里金偏最小二乘法，使此类向量适用于搜索过程。我们提出的方法名为神经线性遗传编程代理模型（NeuroLGP-SM），可在无需完整评估的情况下高效准确估计深度神经网络适应度。与包括未使用代理模型的NeuroLGP、卷积神经网络、支持向量机及自编码器在内的12种方法相比，NeuroLGP-SM展现出具有竞争力甚至更优的结果。值得注意的是，NeuroLGP-SM的能效较其原版NeuroLGP提升了25%。这一效率优势进一步增强了我们提出的NeuroLGP-SM在优化大型深度神经网络配置中的整体吸引力。