Effective communication between the server and workers plays a key role in distributed optimization. In this paper, we focus on optimizing the server-to-worker communication, uncovering inefficiencies in prevalent downlink compression approaches. Considering first the pure setup where the uplink communication costs are negligible, we introduce MARINA-P, a novel method for downlink compression, employing a collection of correlated compressors. Theoretical analyses demonstrates that MARINA-P with permutation compressors can achieve a server-to-worker communication complexity improving with the number of workers, thus being provably superior to existing algorithms. We further show that MARINA-P can serve as a starting point for extensions such as methods supporting bidirectional compression. We introduce M3, a method combining MARINA-P with uplink compression and a momentum step, achieving bidirectional compression with provable improvements in total communication complexity as the number of workers increases. Theoretical findings align closely with empirical experiments, underscoring the efficiency of the proposed algorithms.

翻译：在分布式优化中，服务器与工作节点之间的有效通信至关重要。本文聚焦于优化服务器到工作节点的通信，揭示了现有下行压缩方法中的低效问题。首先考虑上行通信成本可忽略的纯设置，我们提出了一种新颖的下行压缩方法MARINA-P，该方法采用一组相关压缩器。理论分析表明，采用置换压缩器的MARINA-P能够实现随工作节点数量增加而改善的服务器到工作节点通信复杂度，因此被证实优于现有算法。我们进一步证明，MARINA-P可作为扩展方法的起点，例如支持双向压缩的方法。我们提出M3方法，它将MARINA-P与上行压缩及动量步骤相结合，实现了双向压缩，并具有随工作节点数量增加而总通信复杂度可证明改善的特性。理论发现与实证实验结果高度吻合，凸显了所提算法的效率。