Genetic algorithms constitute a family of black-box optimization algorithms, which take inspiration from the principles of biological evolution. While they provide a general-purpose tool for optimization, their particular instantiations can be heuristic and motivated by loose biological intuition. In this work we explore a fundamentally different approach: Given a sufficiently flexible parametrization of the genetic operators, we discover entirely new genetic algorithms in a data-driven fashion. More specifically, we parametrize selection and mutation rate adaptation as cross- and self-attention modules and use Meta-Black-Box-Optimization to evolve their parameters on a set of diverse optimization tasks. The resulting Learned Genetic Algorithm outperforms state-of-the-art adaptive baseline genetic algorithms and generalizes far beyond its meta-training settings. The learned algorithm can be applied to previously unseen optimization problems, search dimensions & evaluation budgets. We conduct extensive analysis of the discovered operators and provide ablation experiments, which highlight the benefits of flexible module parametrization and the ability to transfer (`plug-in') the learned operators to conventional genetic algorithms.

翻译：遗传算法构成了黑箱优化算法的一个家族，其灵感来源于生物进化原理。虽然它们提供了通用的优化工具，但具体的实例化往往是启发式的，并源于宽松的生物学直觉。在本工作中，我们探索了一种根本不同的方法：给定遗传算子的足够灵活的参数化，我们以数据驱动的方式发现全新的遗传算法。具体而言，我们将选择算子和变异率自适应参数化为交叉注意力和自注意力模块，并使用元黑箱优化在一组多样化的优化任务上进化其参数。由此产生的学习遗传算法超越了最先进的自适应基线遗传算法，并远远泛化到其元训练设置之外。学习到的算法可应用于之前未见过的优化问题、搜索维度和评估预算。我们对发现的算子进行了广泛分析，并提供了消融实验，这突显了灵活模块参数化的优势以及将学习到的算子迁移（“即插即用”）到传统遗传算法的能力。