We revisit FedExProx - a recently proposed distributed optimization method designed to enhance convergence properties of parallel proximal algorithms via extrapolation. In the process, we uncover a surprising flaw: its known theoretical guarantees on quadratic optimization tasks are no better than those offered by the vanilla Gradient Descent (GD) method. Motivated by this observation, we develop a novel analysis framework, establishing a tighter linear convergence rate for non-strongly convex quadratic problems. By incorporating both computation and communication costs, we demonstrate that FedExProx can indeed provably outperform GD, in stark contrast to the original analysis. Furthermore, we consider partial participation scenarios and analyze two adaptive extrapolation strategies - based on gradient diversity and Polyak stepsizes - again significantly outperforming previous results. Moving beyond quadratics, we extend the applicability of our analysis to general functions satisfying the Polyak-Lojasiewicz condition, outperforming the previous strongly convex analysis while operating under weaker assumptions. Backed by empirical results, our findings point to a new and stronger potential of FedExProx, paving the way for further exploration of the benefits of extrapolation in federated learning.

翻译：我们重新审视了FedExProx——一种近期提出的分布式优化方法，旨在通过外推法增强并行邻近算法的收敛特性。在此过程中，我们发现了一个令人惊讶的缺陷：其在二次优化任务上的已知理论保证并不优于朴素梯度下降（GD）方法所提供的性能。受此观察启发，我们开发了一种新颖的分析框架，为非强凸二次问题建立了更紧的线性收敛率。通过同时考虑计算和通信成本，我们证明FedExProx确实能够以可证明的方式优于GD，这与原始分析形成鲜明对比。此外，我们考虑了部分参与场景，并分析了两种自适应外推策略——基于梯度多样性和Polyak步长——再次显著优于先前结果。在超越二次函数的情形下，我们将分析推广到满足Polyak-Lojasiewicz条件的一般函数，在更弱的假设下优于先前的强凸分析。在实证结果的支持下，我们的发现揭示了FedExProx新的、更强的潜力，为联邦学习中进一步探索外推法的优势铺平了道路。