This work is concerned with computing low-rank approximations of a matrix function $f(A)$ for a large symmetric positive semi-definite matrix $A$, a task that arises in, e.g., statistical learning and inverse problems. The application of popular randomized methods, such as the randomized singular value decomposition or the Nystr\"om approximation, to $f(A)$ requires multiplying $f(A)$ with a few random vectors. A significant disadvantage of such an approach, matrix-vector products with $f(A)$ are considerably more expensive than matrix-vector products with $A$, even when carried out only approximately via, e.g., the Lanczos method. In this work, we present and analyze funNystr\"om, a simple and inexpensive method that constructs a low-rank approximation of $f(A)$ directly from a Nystr\"om approximation of $A$, completely bypassing the need for matrix-vector products with $f(A)$. It is sensible to use funNystr\"om whenever $f$ is monotone and satisfies $f(0) = 0$. Under the stronger assumption that $f$ is operator monotone, which includes the matrix square root $A^{1/2}$ and the matrix logarithm $\log(I+A)$, we derive probabilistic bounds for the error in the Frobenius, nuclear, and operator norms. These bounds confirm the numerical observation that funNystr\"om tends to return an approximation that compares well with the best low-rank approximation of $f(A)$. Furthermore, compared to existing methods, funNystr\"om requires significantly fewer matrix-vector products with $A$ to obtain a low-rank approximation of $f(A)$, without sacrificing accuracy or reliability. Our method is also of interest when estimating quantities associated with $f(A)$, such as the trace or the diagonal entries of $f(A)$. In particular, we propose and analyze funNystr\"om++, a combination of funNystr\"om with the recently developed Hutch++ method for trace estimation.
翻译:本文关注于为大型对称半正定矩阵$A$计算矩阵函数$f(A)$的低秩近似问题,该问题常见于统计学习与反问题等领域。将流行随机方法(如随机奇异值分解或Nyström近似)应用于$f(A)$需要计算$f(A)$与若干随机向量的乘积。这类方法的一个显著缺陷在于:即使通过Lanczos方法进行近似计算,$f(A)$的矩阵-向量积运算成本仍远高于$A$的对应运算。本文提出并分析funNyström方法——一种简洁且低成本的新方法,它直接从$A$的Nyström近似构建$f(A)$的低秩近似,完全规避了与$f(A)$的矩阵-向量积运算。该方法适用于满足$f(0)=0$的单调函数$f$。在更强的算子单调假设(涵盖矩阵平方根$A^{1/2}$与矩阵对数$\log(I+A)$)下,我们推导了Frobenius范数、核范数和算子范数误差的概率界。这些理论界证实了数值观测结果:funNyström方法倾向于返回与$f(A)$最优低秩近似相媲美的近似结果。进一步,与现有方法相比,funNyström在保持精度与可靠性的前提下,显著减少了获取$f(A)$低秩近似所需的$A$矩阵-向量积运算次数。该方法在估计与$f(A)$相关的量(如迹或对角线元素)时同样具有价值。特别地,我们提出并分析了funNyström++——将funNyström与最新开发的迹估计方法Hutch++相结合的算法。