Sketch-and-precondition techniques are efficient and popular for solving large least squares (LS) problems of the form $Ax=b$ with $A\in\mathbb{R}^{m\times n}$ and $m\gg n$. This is where $A$ is ``sketched" to a smaller matrix $SA$ with $S\in\mathbb{R}^{\lceil cn\rceil\times m}$ for some constant $c>1$ before an iterative LS solver computes the solution to $Ax=b$ with a right preconditioner $P$, where $P$ is constructed from $SA$. Prominent sketch-and-precondition LS solvers are Blendenpik and LSRN. We show that the sketch-and-precondition technique in its most commonly used form is not numerically stable for ill-conditioned LS problems. For provable and practical backward stability and optimal residuals, we suggest using an unpreconditioned iterative LS solver on $(AP)z=b$ with $x=Pz$. Provided the condition number of $A$ is smaller than the reciprocal of the unit round-off, we show that this modification ensures that the computed solution has a backward error comparable to the iterative LS solver applied to a well-conditioned matrix. Using smoothed analysis, we model floating-point rounding errors to argue that our modification is expected to compute a backward stable solution even for arbitrarily ill-conditioned LS problems. Additionally, we provide experimental evidence that using the sketch-and-solve solution as a starting vector in sketch-and-precondition algorithms (as suggested by Rokhlin and Tygert in 2008) should be highly preferred over the zero vector. The initialization often results in much more accurate solutions -- albeit not always backward stable ones.

翻译：草图与预处理技术是求解大规模最小二乘（LS）问题（形式为$Ax=b$，其中$A\in\mathbb{R}^{m\times n}$且$m\gg n$）的高效且流行方法。该方法通过构造草图矩阵$S\in\mathbb{R}^{\lceil cn\rceil\times m}$（常数$c>1$）将$A$压缩为较小矩阵$SA$，并基于$SA$构建右预处理器$P$，随后由迭代LS求解器求解$Ax=b$。典型的草图与预处理LS求解器包括Blendenpik和LSRN。本文证明，在最常用形式下，草图与预处理技术对病态LS问题不具备数值稳定性。为获得可证明且实用的后向稳定性及最优残差，我们建议在$(AP)z=b$（其中$x=Pz$）上使用无预处理的迭代LS求解器。当$A$的条件数小于单位舍入误差的倒数时，该修正能确保计算解的后向误差与应用于良态矩阵的迭代LS求解器相当。通过平滑分析对浮点舍入误差建模，我们论证即使面对任意病态LS问题，该修正方法仍可望计算后向稳定解。此外，实验证据表明：在草图与预处理算法中采用草图-求解解作为初始向量（如Rokhlin与Tygert于2008年建议）应远优于零向量初始化——尽管其解未必始终后向稳定，但往往能显著提升精度。