We introduce a new Swarm-Based Gradient Descent (SBGD) method for non-convex optimization. The swarm consists of agents, each is identified with a position, ${\mathbf x}$, and mass, $m$. The key to their dynamics is communication: masses are being transferred from agents at high ground to low(-est) ground. At the same time, agents change positions with step size, $h=h({\mathbf x},m)$, adjusted to their relative mass: heavier agents proceed with small time-steps in the direction of local gradient, while lighter agents take larger time-steps based on a backtracking protocol. Accordingly, the crowd of agents is dynamically divided between `heavier' leaders, expected to approach local minima, and `lighter' explorers. With their large-step protocol, explorers are expected to encounter improved position for the swarm; if they do, then they assume the role of `heavy' swarm leaders and so on. Convergence analysis and numerical simulations in one-, two-, and 20-dimensional benchmarks demonstrate the effectiveness of SBGD as a global optimizer.

翻译：我们提出了一种新的基于群体的梯度下降方法（SBGD），用于解决非凸优化问题。该群体由多个代理构成，每个代理具有位置向量 ${\mathbf x}$ 和质量 $m$。其动力学核心在于通信机制：质量从地势较高的代理向地势最低的代理转移。与此同时，代理以步长 $h=h({\mathbf x},m)$ 改变位置，该步长根据其相对质量动态调整：质量较大的代理沿局部梯度方向以小步长行进，而质量较小的代理则基于回溯协议采取大步长。由此，代理群体被动态划分为两类：预期趋近局部极小值的"重型"领导者，以及"轻型"探索者。通过其大步长协议，探索者有望为群体找到更优位置；若成功实现，它们将转变为"重型"群体领导者，如此循环往复。在一维、二维及二十维基准问题上的收敛性分析与数值模拟表明，SBGD作为一种全局优化方法具有显著有效性。