We extend our study of the swarm-based gradient descent method for non-convex optimization, [Lu, Tadmor & Zenginoglu, arXiv:2211.17157], to allow random descent directions. We recall that the swarm-based approach consists of a swarm of agents, each identified with a position, ${\mathbf x}$, and mass, $m$. The key is the transfer of mass from high ground to low(-est) ground. The mass of an agent dictates its step size: lighter agents take larger steps. In this paper, the essential new feature is the choice of direction: rather than restricting the swarm to march in the steepest gradient descent, we let agents proceed in randomly chosen directions centered around -- but otherwise different from -- the gradient direction. The random search secures the descent property while at the same time, enabling greater exploration of ambient space. Convergence analysis and benchmark optimizations demonstrate the effectiveness of the swarm-based random descent method as a multi-dimensional global optimizer.

翻译：我们拓展了基于群体智能的非凸优化梯度下降方法研究（Lu, Tadmor & Zenginoglu, arXiv:2211.17157），允许采用随机下降方向。回顾可知，基于群体智能的方法包含一群智能体，每个智能体由位置x和质量m标识。其关键在于将质量从高位区域转移至最（低）低位区域。智能体的质量决定了其步长：较轻的智能体采取更大的步长。本文的核心创新在于方向选择：我们不再限制群体沿最陡梯度下降方向行进，而是允许智能体沿以梯度方向为中心但与其不同的随机选取方向前进。这种随机搜索在保证下降性质的同时，增强了对环境空间的探索能力。收敛性分析与基准优化实验表明，基于群体智能的随机下降方法作为多维全局优化器具有显著有效性。