We develop two compression based stochastic gradient algorithms to solve a class of non-smooth strongly convex-strongly concave saddle-point problems in a decentralized setting (without a central server). Our first algorithm is a Restart-based Decentralized Proximal Stochastic Gradient method with Compression (C-RDPSG) for general stochastic settings. We provide rigorous theoretical guarantees of C-RDPSG with gradient computation complexity and communication complexity of order $\mathcal{O}( (1+\delta)^4 \frac{1}{L^2}{\kappa_f^2}\kappa_g^2 \frac{1}{\epsilon} )$, to achieve an $\epsilon$-accurate saddle-point solution, where $\delta$ denotes the compression factor, $\kappa_f$ and $\kappa_g$ denote respectively the condition numbers of objective function and communication graph, and $L$ denotes the smoothness parameter of the smooth part of the objective function. Next, we present a Decentralized Proximal Stochastic Variance Reduced Gradient algorithm with Compression (C-DPSVRG) for finite sum setting which exhibits gradient computation complexity and communication complexity of order $\mathcal{O} \left((1+\delta) \max \{\kappa_f^2, \sqrt{\delta}\kappa^2_f\kappa_g,\kappa_g \} \log\left(\frac{1}{\epsilon}\right) \right)$. Extensive numerical experiments show competitive performance of the proposed algorithms and provide support to the theoretical results obtained.
翻译:我们提出了两种基于压缩的随机梯度算法,用于解决去中心化环境(无中心服务器)下的一类非光滑强凸-强凹鞍点问题。第一种算法是面向一般随机场景的带压缩的重启式去中心化近端随机梯度方法(C-RDPSG)。我们为C-RDPSG提供了严格的理论保证:为达到ε精度的鞍点解,其梯度计算复杂度与通信复杂度均为$\mathcal{O}( (1+\delta)^4 \frac{1}{L^2}{\kappa_f^2}\kappa_g^2 \frac{1}{\epsilon} )$,其中$\delta$表示压缩因子,$\kappa_f$和$\kappa_g$分别表示目标函数与通信图的条件数,$L$表示目标函数光滑部分的平滑参数。进一步地,我们针对有限和场景提出了带压缩的去中心化近端随机方差缩减梯度算法(C-DPSVRG),其梯度计算复杂度与通信复杂度为$\mathcal{O} \left((1+\delta) \max \{\kappa_f^2, \sqrt{\delta}\kappa^2_f\kappa_g,\kappa_g \} \log\left(\frac{1}{\epsilon}\right) \right)$。大量数值实验表明所提算法具有竞争性能,并验证了理论结果。