Effective communication between the server and workers plays a key role in distributed optimization. In this paper, we focus on optimizing the server-to-worker communication, uncovering inefficiencies in prevalent downlink compression approaches. Considering first the pure setup where the uplink communication costs are negligible, we introduce MARINA-P, a novel method for downlink compression, employing a collection of correlated compressors. Theoretical analyses demonstrates that MARINA-P with permutation compressors can achieve a server-to-worker communication complexity improving with the number of workers, thus being provably superior to existing algorithms. We further show that MARINA-P can serve as a starting point for extensions such as methods supporting bidirectional compression. We introduce M3, a method combining MARINA-P with uplink compression and a momentum step, achieving bidirectional compression with provable improvements in total communication complexity as the number of workers increases. Theoretical findings align closely with empirical experiments, underscoring the efficiency of the proposed algorithms.

翻译：服务器与工作节点间的有效通信在分布式优化中扮演着关键角色。本文聚焦于优化服务器到工作节点的通信，揭示了主流下行链路压缩方法中存在的低效问题。首先考虑上行链路通信成本可忽略的纯设置场景，我们提出了MARINA-P——一种采用一组相关压缩器的新型下行链路压缩方法。理论分析表明，配备置换压缩器的MARINA-P能够实现随工作节点数量增加而提升的服务器到工作节点通信复杂度，从而被证明优于现有算法。我们进一步证明，MARINA-P可作为支持双向压缩等扩展方法的起点。我们提出了M3方法，该方法将MARINA-P与上行链路压缩及动量步骤相结合，实现了双向压缩，并在工作节点增加时，可证明其总通信复杂度得到提升。理论发现与实证实验高度吻合，印证了所提算法的高效性。