Sample efficiency is crucial in optimization, particularly in black-box scenarios characterized by expensive evaluations and zeroth-order feedback. When computing resources are plentiful, Bayesian optimization is often favored over evolution strategies. In this paper, we introduce a full invariance oriented evolution strategies algorithm, derived from its corresponding framework, that effectively rivals the leading Bayesian optimization method in tasks with dimensions at the upper limit of Bayesian capability. Specifically, we first build the framework InvIGO that fully incorporates historical information while retaining the full invariant and computational complexity. We then exemplify InvIGO on multi-dimensional Gaussian, which gives an invariant and scalable optimizer SynCMA . The theoretical behavior and advantages of our algorithm over other Gaussian-based evolution strategies are further analyzed. Finally, We benchmark SynCMA against leading algorithms in Bayesian optimization and evolution strategies on various high dimension tasks, in cluding Mujoco locomotion tasks, rover planning task and synthetic functions. In all scenarios, SynCMA demonstrates great competence, if not dominance, over other algorithms in sample efficiency, showing the underdeveloped potential of property oriented evolution strategies.

翻译：样本效率在优化中至关重要，特别是在评估成本高昂且采用零阶反馈的黑箱场景中。当计算资源充裕时，贝叶斯优化通常比进化策略更受青睐。本文提出了一种基于其对应框架的完全不变性导向的进化策略算法，该算法能在贝叶斯优化能力上限的维度任务中有效媲美领先的贝叶斯优化方法。具体而言，我们首先构建了InvIGO框架，该框架在保持完全不变性和计算复杂度的同时，充分整合了历史信息。随后，我们以多维高斯分布为例对InvIGO进行实例化，得到了具有不变性和可扩展性的优化器SynCMA。进一步分析了该算法相对于其他基于高斯的进化策略的理论优势和行为特性。最后，我们将SynCMA与贝叶斯优化及进化策略中的领先算法，在多种高维任务（包括Mujoco运动控制任务、漫游车规划任务及合成函数）上进行了基准测试。在所有场景中，SynCMA在样本效率上展现出强大竞争力（甚至主导优势），揭示了面向属性的进化策略尚未被充分挖掘的潜力。