While effective in practice, iterative methods for solving large systems of linear equations can be significantly affected by problem-dependent condition number quantities. This makes characterizing their time complexity challenging, particularly when we wish to make comparisons between deterministic and stochastic methods, that may or may not rely on preconditioning and/or fast matrix multiplication. In this work, we consider a fine-grained notion of complexity for iterative linear solvers which we call the spectral tail condition number, $\kappa_\ell$, defined as the ratio between the $\ell$th largest and the smallest singular value of the matrix representing the system. Concretely, we prove the following main algorithmic result: Given an $n\times n$ matrix $A$ and a vector $b$, we can find $\tilde{x}$ such that $\|A\tilde{x}-b\|\leq\epsilon\|b\|$ in time $\tilde{O}(\kappa_\ell\cdot n^2\log 1/\epsilon)$ for any $\ell = O(n^{\frac1{\omega-1}})=O(n^{0.729})$, where $\omega \approx 2.372$ is the current fast matrix multiplication exponent. This guarantee is achieved by Sketch-and-Project with Nesterov's acceleration. Some of the implications of our result, and of the use of $\kappa_\ell$, include direct improvement over a fine-grained analysis of the Conjugate Gradient method, suggesting a stronger separation between deterministic and stochastic iterative solvers; and relating the complexity of iterative solvers to the ongoing algorithmic advances in fast matrix multiplication, since the bound on $\ell$ improves with $\omega$. Our main technical contributions are new sharp characterizations for the first and second moments of the random projection matrix that commonly arises in sketching algorithms, building on a combination of techniques from combinatorial sampling via determinantal point processes and Gaussian universality results from random matrix theory.

翻译：尽管迭代方法在求解大型线性方程组时实践中有效，但其性能会显著受到与问题相关的条件数量的影响。这使得刻画其时间复杂度具有挑战性，特别是当我们希望比较可能依赖或不依赖预处理和/或快速矩阵乘法的确定性与随机性方法时。本文提出了一种用于迭代线性求解器的细粒度复杂度概念，称为谱尾条件数 $\kappa_\ell$，定义为表示系统的矩阵的第 $\ell$ 大奇异值与最小奇异值之比。具体地，我们证明了以下主要算法结果：给定一个 $n\times n$ 矩阵 $A$ 和一个向量 $b$，对于任意 $\ell = O(n^{\frac1{\omega-1}})=O(n^{0.729})$（其中 $\omega \approx 2.372$ 是当前快速矩阵乘法指数），我们可以在 $\tilde{O}(\kappa_\ell\cdot n^2\log 1/\epsilon)$ 时间内找到 $\tilde{x}$，使得 $\|A\tilde{x}-b\|\leq\epsilon\|b\|$。该保证通过使用 Nesterov 加速的 Sketch-and-Project 方法实现。我们的结果以及 $\kappa_\ell$ 的运用包含以下推论：直接改进共轭梯度法的细粒度分析，暗示确定性与随机性迭代求解器之间存在更强的分离；以及将迭代求解器的复杂度与快速矩阵乘法的最新算法进展相关联，因为 $\ell$ 的上界随 $\omega$ 改善。我们的主要技术贡献在于对素描算法中常见的随机投影矩阵的一阶和二阶矩进行了新的精确刻画，这基于组合抽样（通过行列式点过程）与随机矩阵理论中的高斯普适性结果相结合的技巧。