The Kaczmarz algorithm is an iterative method that solves linear systems of equations. It stands out among iterative algorithms when dealing with large systems for two reasons. First, at each iteration, the Kaczmarz algorithm uses a single equation, resulting in minimal computational work per iteration. Second, solving the entire system may only require the use of a small subset of the equations. These characteristics have attracted significant attention to the Kaczmarz algorithm. Researchers have observed that randomly choosing equations can improve the convergence rate of the algorithm. This insight led to the development of the Randomized Kaczmarz algorithm and, subsequently, several other variations emerged. In this paper, we extensively analyze the native Kaczmarz algorithm and many of its variations using large-scale dense random systems as benchmarks. Through our investigation, we have verified that, for consistent systems, various row sampling schemes can outperform both the original and Randomized Kaczmarz method. Specifically, sampling without replacement and using quasirandom numbers are the fastest techniques. However, for inconsistent systems, the Conjugate Gradient method for Least-Squares problems overcomes all variations of the Kaczmarz method for these types of systems.

翻译：Kaczmarz算法是一种解决线性方程组的迭代方法。在处理大规模方程组时，该算法因两大优势在迭代算法中脱颖而出：首先，每次迭代仅使用单个方程，因此每次迭代的计算量极小；其次，求解整个方程组可能只需使用一小部分方程。这些特性使Kaczmarz算法备受关注。研究者发现，随机选择方程可提升算法收敛速度。这一发现催生了随机化Kaczmarz算法，随后又涌现出多种变体。本文以大规模稠密随机系统为基准，对经典Kaczmarz算法及其多种变体进行了全面分析。通过研究，我们验证了在相容系统中，多种行采样方案均可优于原始Kaczmarz方法和随机化Kaczmarz方法。具体而言，无放回采样和拟随机数技术是速度最快的方法。然而，对于不相容系统，最小二乘问题的共轭梯度法在此类系统中优于Kaczmarz方法的所有变体。