In this paper we study consensus-based optimization (CBO), a versatile, flexible and customizable optimization method suitable for performing nonconvex and nonsmooth global optimizations in high dimensions. CBO is a multi-particle metaheuristic, which is effective in various applications and at the same time amenable to theoretical analysis thanks to its minimalistic design. The underlying dynamics, however, is flexible enough to incorporate different mechanisms widely used in evolutionary computation and machine learning, as we show by analyzing a variant of CBO which makes use of memory effects and gradient information. We rigorously prove that this dynamics converges to a global minimizer of the objective function in mean-field law for a vast class of functions under minimal assumptions on the initialization of the method. The proof in particular reveals how to leverage further, in some applications advantageous, forces in the dynamics without loosing provable global convergence. To demonstrate the benefit of the herein investigated memory effects and gradient information in certain applications, we present numerical evidence for the superiority of this CBO variant in applications such as machine learning and compressed sensing, which en passant widen the scope of applications of CBO.

翻译：本文研究共识优化方法（CBO）——一种适用于高维非凸非光滑全局优化的多功能、灵活且可定制的优化方法。CBO是一种多粒子元启发式算法，凭借其极简设计，在各类应用中表现出高效性，同时便于理论分析。然而，其底层动力学框架具有足够灵活性，可整合进化计算与机器学习中广泛使用的多种机制——本文通过分析引入记忆效应与梯度信息的CBO变体验证了这一点。我们严格证明：在极小初始化假设条件下，该动力学过程对广泛函数类在平均场定律下收敛至目标函数全局最小值。该证明尤其揭示了如何在动力学中进一步利用某些应用中具优势的附加力项（如记忆效应与梯度信息），同时保持可证明的全局收敛性。为展示本文研究的记忆效应与梯度信息在特定应用中的优势，我们通过数值实验证明该CBO变体在机器学习与压缩感知等领域的优越性——这同时拓展了CBO的应用范畴。