The Kaczmarz algorithm is an iterative technique designed to solve consistent linear systems of equations. It falls within the category of row-action methods, focusing on handling one equation per iteration. This characteristic makes it especially useful in solving very large systems. The recent introduction of a randomized version, the Randomized Kaczmarz method, renewed interest in the algorithm, leading to the development of numerous variations. Subsequently, parallel implementations for both the original and Randomized Kaczmarz method have since then been proposed. However, previous work has addressed sparse linear systems, whereas we focus on solving dense systems. In this paper, we explore in detail approaches to parallelizing the Kaczmarz method for both shared and distributed memory for large dense systems. In particular, we implemented the Randomized Kaczmarz with Averaging (RKA) method that, for inconsistent systems, unlike the standard Randomized Kaczmarz algorithm, reduces the final error of the solution. While efficient parallelization of this algorithm is not achievable, we introduce a block version of the averaging method that can outperform the RKA method.

翻译：Kaczmarz算法是一种用于求解相容线性方程组的迭代技术。它属于逐行动作方法，每次迭代仅处理一个方程。这一特性使其在求解超大规模系统时尤为实用。近期引入的随机化变体——随机Kaczmarz方法，重新激发了人们对这一算法的研究兴趣，催生了诸多衍生算法。此后，研究者提出了原始Kaczmarz方法及随机Kaczmarz方法的并行实现方案。然而，既有工作主要针对稀疏线性系统，而本文聚焦于求解稠密系统。我们深入探究了在共享内存与分布式内存环境下，针对大型稠密系统并行化Kaczmarz方法的多种实现路径。特别地，我们实现了带平均化的随机Kaczmarz（RKA）方法——对于不相容系统，该方法与标准随机Kaczmarz算法不同，能够降低最终解的误差。尽管该算法难以实现高效并行化，但本文引入了平均化方法的分块版本，其性能可超越RKA方法。