We consider the fundamental problem of solving a large-scale system of linear equations. In particular, we consider the setting where a taskmaster intends to solve the system in a distributed/federated fashion with the help of a set of machines, who each have a subset of the equations. Although there exist several approaches for solving this problem, missing is a rigorous comparison between the convergence rates of the projection-based methods and those of the optimization-based ones. In this paper, we analyze and compare these two classes of algorithms with a particular focus on the most efficient method from each class, namely, the recently proposed Accelerated Projection-Based Consensus (APC) and the Distributed Heavy-Ball Method (D-HBM). To this end, we first propose a geometric notion of data heterogeneity called angular heterogeneity and discuss its generality. Using this notion, we bound and compare the convergence rates of the studied algorithms and capture the effects of both cross-machine and local data heterogeneity on these quantities. Our analysis results in a number of novel insights besides showing that APC is the most efficient method in realistic scenarios where there is a large data heterogeneity. Our numerical analyses validate our theoretical results.

翻译：我们考虑求解大规模线性方程组这一基本问题。具体而言，我们考虑以下场景：一个任务管理者希望以分布式/联邦方式借助一组机器求解方程组，每台机器拥有部分方程。尽管存在多种求解该问题的方法，但目前尚缺乏对基于投影的方法与基于优化的方法在收敛速度上的严格比较。本文分析并比较了这两类算法，重点关注每类中最有效的方法，即最近提出的加速投影一致性法（APC）和分布式重球法（D-HBM）。为此，我们首先提出一种名为角度异构性的几何化数据异构性概念，并讨论其普适性。基于这一概念，我们限定并比较了所研究算法的收敛速度，同时捕捉了跨机器与局部数据异构性对这些量的影响。我们的分析得出了若干新见解，此外还表明，在数据异构性较大的实际场景中，APC是最有效的方法。数值分析验证了我们的理论结果。