Inspired by the success of performing multiple local optimization steps between communication rounds in federated learning, incorporating such local updates into distributed optimization has recently attracted growing interest. However, unlike federated learning, where local updates can accelerate training by reducing gradient estimation error under minibatch settings, it remains unclear whether similar benefits persist when exact gradients are available. Moreover, existing theoretical results typically require reducing the step size when multiple local updates are employed, which can entirely offset any potential benefit of these additional local updates. In this paper, we focus on the classic DIGing algorithm and leverage the tight performance bounds provided by Performance Estimation Problems (PEP) to show that incorporating local updates can indeed accelerate distributed optimization. To the best of our knowledge, this is the first rigorous demonstration of such acceleration for a broad class of objective functions. Our analysis further reveals that, under an appropriate step size, performing only two local updates is sufficient to achieve the maximal possible improvement, and that additional local updates provide no further gains. Because more updates increase computational cost, these findings offer practical guidance for efficient implementation. We also show that these speed gains depend critically on the network structure, with sparser or less connected graphs, characterized by the spectral properties of the mixing matrix, yielding smaller improvements. Extensive experiments on both synthetic and real-world datasets corroborate the theoretical findings.

翻译：受联邦学习中通信轮次之间执行多次局部优化步骤成功的启发，将此类局部更新纳入分布式优化近年来引起了越来越多的关注。然而，与联邦学习不同，在联邦学习中，局部更新可以通过减少小批量设置下的梯度估计误差来加速训练，对于当精确梯度可用时，类似好处是否仍然存在仍不清楚。此外，现有的理论结果通常要求在使用多个局部更新时减少步长，这可能会完全抵消这些额外局部更新的任何潜在好处。本文中，我们专注于经典的DIGing算法，并利用性能估计问题（PEP）提供的紧致性能界来证明，引入局部更新确实可以加速分布式优化。据我们所知，这是首次针对广泛目标函数类严格证明这种加速效果。我们的分析进一步表明，在适当的步长下，仅执行两次局部更新就足以实现最大可能的改进，而额外的局部更新不会带来进一步的增益。由于更多更新会增加计算成本，这些发现为高效实现提供了实践指导。我们还表明，这些速度增益关键依赖于网络结构，以混合矩阵谱特性为特征的更稀疏或连通性较差的图产生的改进较小。在合成数据集和真实数据集上的大量实验验证了理论发现。