Neuroevolution has greatly promoted Deep Neural Network (DNN) architecture design and its applications, while there is a lack of methods available across different DNN types concerning both their scale and performance. In this study, we propose a self-adaptive neuroevolution (SANE) approach to automatically construct various lightweight DNN architectures for different tasks. One of the key settings in SANE is the search space defined by cells and organs self-adapted to different DNN types. Based on this search space, a constructive evolution strategy with uniform evolution settings and operations is designed to grow DNN architectures gradually. SANE is able to self-adaptively adjust evolution exploration and exploitation to improve search efficiency. Moreover, a speciation scheme is developed to protect evolution from early convergence by restricting selection competition within species. To evaluate SANE, we carry out neuroevolution experiments to generate different DNN architectures including convolutional neural network, generative adversarial network and long short-term memory. The results illustrate that the obtained DNN architectures could have smaller scale with similar performance compared to existing DNN architectures. Our proposed SANE provides an efficient approach to self-adaptively search DNN architectures across different types.

翻译：神经进化技术极大推动了深度神经网络架构设计及其应用，但目前缺乏能够兼顾不同深度神经网络类型在规模与性能上的通用方法。本研究提出一种自适应性神经进化方法，用于为不同任务自动构建多种轻量级深度神经网络架构。该方法的核心在于定义了由细胞和器官组成的搜索空间，可自适应适配不同类型的深度神经网络。基于该搜索空间，我们设计了一种具有统一进化设置与操作的构造性进化策略，逐步生长深度神经网络架构。该方法能够自适应调整进化探索与利用的平衡，从而提升搜索效率。此外，还开发了物种形成机制，通过限制种内选择竞争来防止进化过早收敛。为评估本方法，我们开展了卷积神经网络、生成对抗网络及长短期记忆网络等不同深度神经网络架构的神经进化实验。结果表明，与现有深度神经网络架构相比，所获网络架构在保持相似性能的同时具有更小规模。本研究提出的自适应性神经进化方法为跨类型深度神经网络架构自适应搜索提供了高效途径。