In this paper, we propose an algorithmic framework to automatically generate efficient deep neural networks and optimize their associated hyperparameters. The framework is based on evolving directed acyclic graphs (DAGs), defining a more flexible search space than the existing ones in the literature. It allows mixtures of different classical operations: convolutions, recurrences and dense layers, but also more newfangled operations such as self-attention. Based on this search space we propose neighbourhood and evolution search operators to optimize both the architecture and hyper-parameters of our networks. These search operators can be used with any metaheuristic capable of handling mixed search spaces. We tested our algorithmic framework with an evolutionary algorithm on a time series prediction benchmark. The results demonstrate that our framework was able to find models outperforming the established baseline on numerous datasets.

翻译：本文提出了一种算法框架，用于自动生成高效的深度神经网络并优化其相关超参数。该框架基于有向无环图（DAG）的演化机制，定义了比现有文献更灵活的搜索空间，允许混合使用不同类型的经典运算（卷积、循环和全连接层）以及自注意力等新型操作。基于该搜索空间，我们提出了邻域搜索与进化搜索算子，以同时优化网络架构与超参数。这些搜索算子可适用于任意能够处理混合搜索空间的元启发式算法。我们在时间序列预测基准上使用进化算法对框架进行了测试，结果表明该框架能够在多个数据集上找到超越既定基线的模型。