This paper investigates distributed zeroth-order optimization for smooth nonconvex problems. We propose a novel variance-reduced gradient estimator, which randomly renovates one orthogonal direction of the true gradient in each iteration while leveraging historical snapshots for variance correction. By integrating this estimator with gradient tracking mechanism, we address the trade-off between convergence rate and sampling cost per zeroth-order gradient estimation that exists in current zeroth-order distributed optimization algorithms, which rely on either the 2-point or $2d$-point gradient estimators. We derive a convergence rate of $\mathcal{O}(d^{\frac{5}{2}}/m)$ for smooth nonconvex functions in terms of sampling number $m$ and problem dimension $d$. Numerical simulations comparing our algorithm with existing methods confirm the effectiveness and efficiency of the proposed gradient estimator.

翻译：本文研究光滑非凸问题的分布式零阶优化。我们提出一种新颖的方差缩减梯度估计器，该估计器在每次迭代中随机更新真实梯度的一个正交方向，同时利用历史快照进行方差校正。通过将该估计器与梯度跟踪机制相结合，我们解决了当前依赖2点或$2d$点梯度估计器的零阶分布式优化算法中存在的收敛速度与每次零阶梯度估计采样成本之间的权衡问题。对于光滑非凸函数，我们推导出$\mathcal{O}(d^{\frac{5}{2}}/m)$的收敛速度，其中$m$为采样次数，$d$为问题维度。通过数值模拟将本算法与现有方法进行比较，结果验证了所提梯度估计器的有效性和高效性。