We consider distributed optimization over a $d$-dimensional space, where $K$ remote clients send coded gradient estimates over an {\em additive Gaussian Multiple Access Channel (MAC)} with noise variance $\sigma_z^2$. Furthermore, the codewords from the clients must satisfy the average power constraint $P$, resulting in a signal-to-noise ratio (SNR) of $KP/\sigma_z^2$. In this paper, we study the fundamental limits imposed by MAC on the {convergence rate of any distributed optimization algorithm and design optimal communication schemes to achieve these limits.} Our first result is a lower bound for the convergence rate, showing that communicating over a MAC imposes a slowdown of $\sqrt{d/\frac{1}{2}\log(1+\SNR)}$ on any protocol compared to the centralized setting. Next, we design a computationally tractable {digital} communication scheme that matches the lower bound to a logarithmic factor in $K$ when combined with a projected stochastic gradient descent algorithm. At the heart of our communication scheme is carefully combining several compression and modulation ideas such as quantizing along random bases, {\em Wyner-Ziv compression}, {\em modulo-lattice decoding}, and {\em amplitude shift keying.} We also show that analog schemes, which are popular due to their ease of implementation, can give close to optimal convergence rates at low $\SNR$ but experience a slowdown of roughly $\sqrt{d}$ at high $\SNR$.

翻译：我们考虑在$d$维空间上的分布式优化问题，其中$K$个远程客户端通过噪声方差为$\sigma_z^2$的加性高斯多址接入信道发送编码梯度估计。此外，客户端的码字必须满足平均功率约束$P$，从而得到信噪比为$KP/\sigma_z^2$。本文研究多址接入信道对任何分布式优化算法收敛速度施加的基本极限，并设计最优通信方案以实现这些极限。我们的第一个结果是收敛速度的下界，表明与集中式设置相比，通过多址接入信道进行通信会使任何协议产生$\sqrt{d/\frac{1}{2}\log(1+\SNR)}$的减速。接着，我们设计了一种计算可行的数字通信方案，当与投影随机梯度下降算法结合时，该方案与下界相差不超过$K$的对数因子。该通信方案的核心在于巧妙结合多种压缩与调制思想，例如基于随机基的量化、Wyner-Ziv压缩、模格解码以及幅移键控。我们还证明，由于易于实现而流行的模拟方案在低信噪比下可接近最优收敛速度，但在高信噪比下会经历大约$\sqrt{d}$倍的减速。