We consider the fundamental problem of solving a large-scale system of linear equations. In particular, we consider the setting where a taskmaster intends to solve the system in a distributed/federated fashion with the help of a set of machines, who each have a subset of the equations. Although there exist several approaches for solving this problem, missing is a rigorous comparison between the convergence rates of the projection-based methods and those of the optimization-based ones. In this paper, we analyze and compare these two classes of algorithms with a particular focus on the most efficient method from each class, namely, the recently proposed Accelerated Projection-Based Consensus (APC) and the Distributed Heavy-Ball Method (D-HBM). To this end, we first propose a geometric notion of data heterogeneity called angular heterogeneity and discuss its generality. Using this notion, we bound and compare the convergence rates of the studied algorithms and capture the effects of both cross-machine and local data heterogeneity on these quantities. Our analysis results in a number of novel insights besides showing that APC is the most efficient method in realistic scenarios where there is a large data heterogeneity. Our numerical analyses validate our theoretical results.

翻译：我们考虑求解大规模线性方程组这一基本问题。具体而言，我们研究如下设定：一个任务主控器希望借助一组机器以分布式/联邦方式求解该系统，每台机器拥有方程组的子集。尽管已有多种方法可解决此问题，但尚缺乏对基于投影的方法与基于优化的方法在收敛速度上的严格比较。本文分析并比较了这两类算法，特别关注其中各自最高效的方法，即近期提出的加速投影一致性方法（APC）和分布式重球法（D-HBM）。为此，我们首先提出一种称为角度异质性的数据异质性几何概念，并讨论其普适性。利用该概念，我们对所研究算法的收敛速度进行界定与比较，并捕捉跨机器和局部数据异质性对这些量的影响。我们的分析得出一系列新颖见解，此外还表明APC在数据异质性较大的实际场景中是最高效的方法。数值分析验证了我们的理论结果。