We present a new class of preconditioned iterative methods for solving linear systems of the form $Ax = b$. Our methods are based on constructing a low-rank Nystr\"om approximation to $A$ using sparse random sketching. This approximation is used to construct a preconditioner, which itself is inverted quickly using additional levels of random sketching and preconditioning. We prove that the convergence of our methods depends on a natural average condition number of $A$, which improves as the rank of the Nystr\"om approximation increases. Concretely, this allows us to obtain faster runtimes for a number of fundamental linear algebraic problems: 1. We show how to solve any $n\times n$ linear system that is well-conditioned except for $k$ outlying large singular values in $\tilde{O}(n^{2.065} + k^\omega)$ time, improving on a recent result of [Derezi\'nski, Yang, STOC 2024] for all $k \gtrsim n^{0.78}$. 2. We give the first $\tilde{O}(n^2 + {d_\lambda}^{\omega}$) time algorithm for solving a regularized linear system $(A + \lambda I)x = b$, where $A$ is positive semidefinite with effective dimension $d_\lambda$. This problem arises in applications like Gaussian process regression. 3. We give faster algorithms for approximating Schatten $p$-norms and other matrix norms. For example, for the Schatten 1 (nuclear) norm, we give an algorithm that runs in $\tilde{O}(n^{2.11})$ time, improving on an $\tilde{O}(n^{2.18})$ method of [Musco et al., ITCS 2018]. Interestingly, previous state-of-the-art algorithms for most of the problems above relied on stochastic iterative methods, like stochastic coordinate and gradient descent. Our work takes a completely different approach, instead leveraging tools from matrix sketching.

翻译：我们提出一类新的预处理迭代方法，用于求解形如 $Ax = b$ 的线性系统。该方法基于利用稀疏随机草图技术构造 $A$ 的低秩 Nyström 近似，并以此构造预处理子，再通过额外层次的随机草图与预处理快速求逆。我们证明，此类方法的收敛性取决于 $A$ 的自然平均条件数，且该条件数随 Nyström 近似秩的增加而改善。具体而言，这使我们能在多个基础线性代数问题中获得更快的运行时间：1. 我们证明，对于任意 $n\times n$ 且除 $k$ 个离群大奇异值外病态程度良好的线性系统，可在 $\tilde{O}(n^{2.065} + k^\omega)$ 时间内求解，改进了 [Dereziński, Yang, STOC 2024] 对所有 $k \gtrsim n^{0.78}$ 的最新结果。2. 我们首次给出 $\tilde{O}(n^2 + {d_\lambda}^{\omega}$) 时间算法求解正则化线性系统 $(A + \lambda I)x = b$，其中 $A$ 为半正定矩阵且有效维数为 $d_\lambda$。该问题常见于高斯过程回归等应用。3. 我们提出更快算法来逼近 Schatten $p$-范数及其他矩阵范数。例如，对于 Schatten 1（核）范数，我们给出运行时间为 $\tilde{O}(n^{2.11})$ 的算法，改进了 [Musco et al., ITCS 2018] 中 $\tilde{O}(n^{2.18})$ 的方法。值得注意的是，上述问题的大多数先前最优算法均依赖于随机迭代方法（如随机坐标下降与随机梯度下降）。本研究则另辟蹊径，转而利用矩阵草图技术工具。