We introduce a novel method for non-convex optimization which is at the interface between the swarm-based gradient-descent (SBGD) [J. Lu et. al., ArXiv:2211.17157; E.Tadmor and A. Zenginoglu, Acta Applicandae Math., 190, 2024] and Simulated Annealing (SA) [V. Cerny, J. optimization theory and appl., 45:41-51, 1985; S.Kirkpatrick et. al., Science, 220(4598):671-680, 1983; S. Geman and C.-R. Hwang, SIAM J. Control and Optimization, 24(5):1031-1043, 1986]. We follow the methodology of SBGD in which a swarm of agents, each identified with a position, ${\mathbf x}$ and mass $m$, explores the ambient space. The agents proceed in gradient descent direction, and are subject to Brownian motion with annealing-rate dictated by a decreasing function of their mass. Thus, instead of the SA protocol for time-decreasing temperature, we let the swarm decide how to `cool down' agents, depending on their accumulated mass over time. The dynamics of masses is coupled with the dynamics of positions: agents at higher ground transfer (part of) their mass to those at lower ground. Consequently, the swarm is dynamically divided between heavier, cooler agents viewed as `leaders' and lighter, warmer agents viewed as `explorers'. Mean-field convergence analysis and benchmark optimizations demonstrate the effectiveness of the swarm-based method as a multi-dimensional global optimizer.

翻译：我们提出了一种新颖的非凸优化方法，该方法融合了基于群体的梯度下降（SBGD）[J. Lu 等，arXiv:2211.17157；E. Tadmor 和 A. Zenginoglu，Acta Applicandae Math.，190, 2024] 与模拟退火（SA）[V. Cerny，J. optimization theory and appl.，45:41-51, 1985；S. Kirkpatrick 等，Science，220(4598):671-680, 1983；S. Geman 和 C.-R. Hwang，SIAM J. Control and Optimization，24(5):1031-1043, 1986] 的思想。我们沿用SBGD的方法论：一个由智能体组成的群体（每个智能体由位置 ${\mathbf x}$ 和质量 $m$ 标识）在环境空间中探索。这些智能体沿梯度下降方向运动，并受到布朗运动的影响，其退火速率由质量递减函数决定。因此，与模拟退火中随时间递减温度的控制协议不同，我们让群体根据各智能体随时间累积的质量自行决定如何“冷却”它们。质量的动态与位置的动态相互耦合：处于较高位置的智能体将其（部分）质量转移给较低位置的智能体。最终，群体被动态划分为两类：质量较大、温度较低的“领导者”，以及质量较小、温度较高的“探索者”。平均场收敛分析与基准优化实验表明，该方法作为一种多维全局优化器具有良好效能。