Two widely used randomized algorithms are the sketch-and-solve method for least-squares regression and the randomized SVD for low-rank approximation. These algorithms apply a random embedding to compress a target matrix, and they perform computations on the compressed matrix to save computational cost. This paper asks, what is the optimal random embedding in these algorithms? Also, what is the sharpest possible error bound for the optimal embedding? The paper proves that a random orthonormal matrix is minimax optimal for the sketch-and-solve algorithm while any rotation-invariant embedding is minimax optimal for the randomized SVD. Following these results, the paper obtains the best possible error bounds for sketched least-squares and the randomized SVD. Last, empirical experiments provide evidence of universality phenomena, in which several random embeddings lead to similar accuracy to the optimal embeddings in practice.

翻译：两种广泛使用的随机算法分别是用于最小二乘回归的草图-求解方法和用于低秩逼近的随机SVD。这些算法通过随机嵌入压缩目标矩阵，并对压缩后的矩阵进行计算以节省计算成本。本文提出以下问题：在这些算法中，何种随机嵌入是最优的？此外，对于最优嵌入，其误差界的最小可能值是多少？本文证明，在草图-求解算法中，随机正交矩阵是极小化极大最优的；而对于随机SVD，任何旋转不变嵌入均是极小化极大最优的。基于这些结果，本文获得了草图最小二乘和随机SVD的最佳可能误差界。最后，实验验证了普适性现象：在实际应用中，多种随机嵌入的精度均能接近最优嵌入的精度。