We consider the problem of solving a large-scale system of linear equations in a distributed or federated manner by a taskmaster and a set of machines, each possessing a subset of the equations. We provide a comprehensive comparison of two well-known classes of algorithms used to solve this problem: projection-based methods and optimization-based methods. First, we introduce a novel geometric notion of data heterogeneity called angular heterogeneity and discuss its generality. Using this notion, we characterize the optimal convergence rates of the most prominent algorithms from each class, capturing the effects of the number of machines, the number of equations, and that of both cross-machine and local data heterogeneity on these rates. Our analysis establishes the superiority of Accelerated Projected Consensus in realistic scenarios with significant data heterogeneity and offers several insights into how angular heterogeneity affects the efficiency of the methods studied. Additionally, we develop distributed algorithms for the efficient computation of the proposed angular heterogeneity metrics. Our extensive numerical analyses validate and complement our theoretical results.

翻译：本文研究通过任务主节点与一组机器（每台机器拥有部分方程）分布式或联邦式求解大规模线性方程组的问题。我们对解决该问题的两类著名算法进行了全面比较：基于投影的方法和基于优化的方法。首先，我们提出了一种称为角度异构性的数据异构性几何新概念，并讨论了其普适性。利用这一概念，我们刻画了每类算法中最主要算法的最优收敛速度，揭示了机器数量、方程数量以及跨机器与本地数据异构性对这些速度的影响。我们的分析确立了在数据异构性显著的实际场景中加速投影共识算法的优越性，并就角度异构性如何影响所研究方法的效率提供了若干见解。此外，我们开发了用于高效计算所提角度异构性度量的分布式算法。大量数值分析验证并补充了我们的理论结果。