The GMRES algorithm of Saad and Schultz (1986) is an iterative method for approximately solving linear systems $A{\bf x}={\bf b}$, with initial guess ${\bf x}_0$ and residual ${\bf r}_0 = {\bf b} - A{\bf x}_0$. The algorithm employs the Arnoldi process to generate the Krylov basis vectors (the columns of $V_k$). It is well known that this process can be viewed as a $QR$ factorization of the matrix $B_k = [\: {\bf r}_0, AV_k\:]$ at each iteration. Despite an ${O}(\epsilon)\kappa(B_k)$ loss of orthogonality, for unit roundoff $\epsilon$ and condition number $\kappa$, the modified Gram-Schmidt formulation was shown to be backward stable in the seminal paper by Paige et al. (2006). We present an iterated Gauss-Seidel formulation of the GMRES algorithm (IGS-GMRES) based on the ideas of Ruhe (1983) and \'{S}wirydowicz et al. (2020). IGS-GMRES maintains orthogonality to the level ${O}(\epsilon)\kappa(B_k)$ or ${O}(\epsilon)$, depending on the choice of one or two iterations; for two Gauss-Seidel iterations, the computed Krylov basis vectors remain orthogonal to working precision and the smallest singular value of $V_k$ remains close to one. The resulting GMRES method is thus backward stable. We show that IGS-GMRES can be implemented with only a single synchronization point per iteration, making it relevant to large-scale parallel computing environments. We also demonstrate that, unlike MGS-GMRES, in IGS-GMRES the relative Arnoldi residual corresponding to the computed approximate solution no longer stagnates above machine precision even for highly non-normal systems.

翻译：Saad和Schultz（1986）提出的GMRES算法是一种迭代方法，用于近似求解线性系统$A{\bf x}={\bf b}$，初始猜测为${\bf x}_0$，残差为${\bf r}_0 = {\bf b} - A{\bf x}_0$。该算法利用Arnoldi过程生成Krylov基向量（即$V_k$的列）。众所周知，该过程可视为每次迭代中对矩阵$B_k = [\: {\bf r}_0, AV_k\:]$进行$QR$分解。尽管在单位舍入误差$\epsilon$和条件数$\kappa$下存在${O}(\epsilon)\kappa(B_k)$的正交性损失，但Paige等人（2006）的开创性论文表明，修正的Gram-Schmidt公式是向后稳定的。我们基于Ruhe（1983）和Świrydowicz等人（2020）的思想，提出了GMRES算法的迭代高斯-赛德尔公式（IGS-GMRES）。根据选择一次或两次迭代，IGS-GMRES可将正交性维持在${O}(\epsilon)\kappa(B_k)$或${O}(\epsilon)$水平；对于两次高斯-赛德尔迭代，计算得到的Krylov基向量保持工作精度的正交性，且$V_k$的最小奇异值接近1。因此，所得的GMRES方法是向后稳定的。我们证明，IGS-GMRES每次迭代仅需单个同步点即可实现，使其适用于大规模并行计算环境。我们还证明，与MGS-GMRES不同，在IGS-GMRES中，即使对于高度非正规系统，对应于计算近似解的相对Arnoldi残差不再停滞在机器精度以上。