Approximating the action of a matrix function $f(\mathbf{A})$ on a vector $\mathbf{b}$ is an increasingly important primitive in machine learning, data science, and statistics, with applications such as sampling high dimensional Gaussians, Gaussian process regression and Bayesian inference, principle component analysis, and approximating Hessian spectral densities. Over the past decade, a number of algorithms enjoying strong theoretical guarantees have been proposed for this task. Many of the most successful belong to a family of algorithms called Krylov subspace methods. Remarkably, a classic Krylov subspace method, called the Lanczos method for matrix functions (Lanczos-FA), frequently outperforms newer methods in practice. Our main result is a theoretical justification for this finding: we show that, for a natural class of rational functions, Lanczos-FA matches the error of the best possible Krylov subspace method up to a multiplicative approximation factor. The approximation factor depends on the degree of $f(x)$'s denominator and the condition number of $\mathbf{A}$, but not on the number of iterations $k$. Our result provides a strong justification for the excellent performance of Lanczos-FA, especially on functions that are well approximated by rationals, such as the matrix square root.

翻译：在机器学习、数据科学和统计学中，近似计算矩阵函数 $f(\mathbf{A})$ 对向量 $\mathbf{b}$ 的作用日益成为一种重要的基础运算，其应用包括高维高斯采样、高斯过程回归与贝叶斯推断、主成分分析以及海森矩阵谱密度近似等。过去十年间，已有多种具备严格理论保证的算法被提出以解决该问题。其中最为成功的算法大多属于一类称为Krylov子空间方法的方法族。值得注意的是，一种经典的Krylov子空间方法——针对矩阵函数的Lanczos方法（Lanczos-FA）在实践中经常优于新近提出的方法。本文的主要成果是对这一现象的理论解释：我们证明，对于一类自然的有理函数，Lanczos-FA在误差上能够达到最优Krylov子空间方法的水平，仅相差一个乘法近似因子。该近似因子取决于 $f(x)$ 分母的次数及 $\mathbf{A}$ 的条件数，而与迭代次数 $k$ 无关。我们的研究结果为Lanczos-FA的优异性能——特别在如矩阵平方根这类可由有理函数良好近似的函数上——提供了强有力的理论依据。