Federated learning (FL) is a subfield of machine learning where multiple clients try to collaboratively learn a model over a network under communication constraints. We consider finite-sum federated optimization under a second-order function similarity condition and strong convexity, and propose two new algorithms: SVRP and Catalyzed SVRP. This second-order similarity condition has grown popular recently, and is satisfied in many applications including distributed statistical learning and differentially private empirical risk minimization. The first algorithm, SVRP, combines approximate stochastic proximal point evaluations, client sampling, and variance reduction. We show that SVRP is communication efficient and achieves superior performance to many existing algorithms when function similarity is high enough. Our second algorithm, Catalyzed SVRP, is a Catalyst-accelerated variant of SVRP that achieves even better performance and uniformly improves upon existing algorithms for federated optimization under second-order similarity and strong convexity. In the course of analyzing these algorithms, we provide a new analysis of the Stochastic Proximal Point Method (SPPM) that might be of independent interest. Our analysis of SPPM is simple, allows for approximate proximal point evaluations, does not require any smoothness assumptions, and shows a clear benefit in communication complexity over ordinary distributed stochastic gradient descent.

翻译：联邦学习（FL）是机器学习的一个子领域，其中多个客户端在通信约束下尝试通过网络协作学习一个模型。我们考虑了在二阶函数相似性条件和强凸性下的有限和联邦优化问题，并提出了两种新算法：SVRP 和催化 SVRP。这种二阶相似性条件近年来逐渐流行，并在许多应用中得以满足，包括分布式统计学习和差分隐私经验风险最小化。第一种算法 SVRP 结合了近似随机近端点评估、客户端采样和方差缩减。我们证明，当函数相似性足够高时，SVRP 具有通信效率，并且性能优于许多现有算法。第二种算法催化 SVRP 是 SVRP 的 Catalyst 加速变体，其性能更优，并一致性地改进了现有算法在二阶相似性和强凸性下的联邦优化。在分析这些算法的过程中，我们提供了随机近端点方法（SPPM）的一种新分析，该分析可能具有独立的研究价值。我们对 SPPM 的分析简洁，允许近似近端点评估，无需任何光滑性假设，并显示出在通信复杂度上相较于普通分布式随机梯度下降的明显优势。