The quest for an algorithm that solves an $n\times n$ linear system in $O(n^2)$ time complexity, or $O(n^2 \text{poly}(1/ε))$ when solving up to $ε$ relative error, is a long-standing open problem in numerical linear algebra and theoretical computer science. There are two predominant paradigms for measuring relative error: forward error (i.e., distance from the output to the optimum solution) and backward error (i.e., distance to the nearest problem solved by the output). In most prior studies, convergence of iterative linear system solvers is measured via various notions of forward error, and as a result, depends heavily on the conditioning of the input. Yet, the numerical analysis literature has long advocated for backward error as the more practically relevant notion of approximation. In this work, we show that -- surprisingly -- the classical and simple Richardson iteration incurs at most $1/k$ (relative) backward error after $k$ iterations on any positive semidefinite (PSD) linear system, irrespective of its condition number. This universal convergence rate implies an $O(n^2/ε)$ complexity algorithm for solving a PSD linear system to $ε$ backward error, and we establish similar or better complexity when using a variety of Krylov solvers beyond Richardson. Then, by directly minimizing backward error over a Krylov subspace, we attain an even faster $O(1/k^2)$ universal rate, and we turn this into an efficient algorithm, MINBERR, with complexity $O(n^2/\sqrtε)$. Finally, we extend this approach via normal equations to solving general linear systems in $O(n^2\log(n)/ε)$ time complexity. We report strong numerical performance of our algorithms on benchmark problems.

翻译：寻找一种算法，能以 $O(n^2)$ 时间复杂度求解 $n\times n$ 线性系统，或当求解至 $\epsilon$ 相对误差时达到 $O(n^2 \text{poly}(1/\epsilon))$ 复杂度，是数值线性代数和理论计算机科学中一个长期未解决的开放问题。度量相对误差有两种主要范式：前向误差（即输出与最优解之间的距离）和后向误差（即输出所能精确求解的最近问题与原始问题之间的距离）。在以往的大多数研究中，迭代线性求解器的收敛性是通过前向误差的不同定义来衡量的，因此高度依赖于输入问题的条件数。然而，数值分析文献长期以来一直主张后向误差是更具实际相关性的近似度量。在本工作中，我们证明——令人惊讶的是——经典的简单理查森迭代法在任意半正定线性系统上迭代 $k$ 次后，无论其条件数如何，后向误差至多为 $1/k$（相对值）。这一通用收敛速率意味着存在一个 $O(n^2/\epsilon)$ 复杂度的算法，可将半正定线性系统求解至 $\epsilon$ 后向误差；而对于除理查森法之外的各种Krylov子空间求解器，我们也建立了类似或更优的复杂度。随后，通过直接在Krylov子空间上最小化后向误差，我们实现了更快的通用速率 $O(1/k^2)$，并将其转化为高效算法MINBERR，其复杂度为 $O(n^2/\sqrt{\epsilon})$。最后，我们通过正规方程将该方法推广至一般线性系统，实现 $O(n^2\log(n)/\epsilon)$ 时间复杂度。我们在基准问题上的数值实验结果表明所提算法性能优异。