We propose a new gradient descent algorithm with added stochastic terms for finding the global optimizers of nonconvex optimization problems. A key component in the algorithm is the adaptive tuning of the randomness based on the value of the objective function. In the language of simulated annealing, the temperature is state-dependent. With this, we prove the global convergence of the algorithm with an algebraic rate both in probability and in the parameter space. This is a significant improvement over the classical rate from using a more straightforward control of the noise term. The convergence proof is based on the actual discrete setup of the algorithm, not just its continuous limit as often done in the literature. We also present several numerical examples to demonstrate the efficiency and robustness of the algorithm for reasonably complex objective functions.

翻译：我们提出了一种带有随机项的新梯度下降算法，用于寻找非凸优化问题的全局最优解。该算法的关键组成部分是基于目标函数值对随机性进行自适应调整。用模拟退火的术语来说，温度是状态相关的。基于此，我们证明了该算法在概率和参数空间中以代数速率全局收敛。与采用更直接的噪声项控制方法相比，这相较于经典速率有了显著改进。收敛性证明基于算法的实际离散设置，而非像文献中常见的那样仅依赖于其连续极限。我们还给出了几个数值示例，以展示该算法在处理相当复杂的目标函数时的效率和鲁棒性。