In the post-Moore era, main performance gains of black-box optimizers are increasingly depending on parallelism, especially for large-scale optimization (LSO). Here we propose to parallelize the well-established covariance matrix adaptation evolution strategy (CMA-ES) and in particular its one latest LSO variant called limited-memory CMA-ES (LM-CMA). To achieve efficiency while approximating its powerful invariance property, we present a multilevel learning-based meta-framework for distributed LM-CMA. Owing to its hierarchically organized structure, Meta-ES is well-suited to implement our distributed meta-framework, wherein the outer-ES controls strategy parameters while all parallel inner-ESs run the serial LM-CMA with different settings. For the distribution mean update of the outer-ES, both the elitist and multi-recombination strategy are used in parallel to avoid stagnation and regression, respectively. To exploit spatiotemporal information, the global step-size adaptation combines Meta-ES with the parallel cumulative step-size adaptation. After each isolation time, our meta-framework employs both the structure and parameter learning strategy to combine aligned evolution paths for CMA reconstruction. Experiments on a set of large-scale benchmarking functions with memory-intensive evaluations, arguably reflecting many data-driven optimization problems, validate the benefits (e.g., effectiveness w.r.t. solution quality, and adaptability w.r.t. second-order learning) and costs of our meta-framework.

翻译：在后摩尔时代，黑箱优化器性能提升主要依赖于并行化，尤其是在大规模优化（LSO）中。本文提出对成熟的协方差矩阵自适应演化策略（CMA-ES）及其最新的大规模变体——有限内存CMA-ES（LM-CMA）进行并行化处理。为在逼近其强大不变性特性的同时实现高效性，我们提出一种基于多层级学习的分布式LM-CMA元框架。得益于其分层组织结构，Meta-ES非常适合实现我们的分布式元框架——外层ES控制策略参数，而所有并行的内层ES则运行具有不同设置的串行LM-CMA。在外层ES的分布均值更新中，并行采用精英策略与多重组策略分别避免停滞和退化。为利用时空信息，全局步长自适应将Meta-ES与并行累积步长自适应相结合。每次隔离时间后，我们的元框架通过结构与参数学习策略，合并对齐的演化路径用于CMA重构。在内存密集型评估的大规模基准函数集合（该集合可反映众多数据驱动优化问题）上的实验验证了本元框架的收益（例如解质量的有效性及二阶学习的适应性）与代价。