The title of this paper is motivated by the title of the paper by Forsythe written in 1952 and published in Bull. Amer. Math. Soc., 59 (1953), pp. 299-329. Forsythe argues that solving a system of $n$ linear algebraic equations in $n$ unknowns is mathematically a lowly subject. His beautiful text graduates with what was at that time ``the newest process on the roster, the method of conjugate gradients.'' We consider it important to revisit, after 70 years, to what extent Forsythe's views, and the views presented in the related contemporary works of Hestenes, Stiefel, Lanczos, Karush, and Hayes, remain relevant today. Including, besides the conjugate gradient method (CG), also the generalized minimal residual method (GMRES), we point out building blocks that we consider central for the current mathematical and computational understanding of Krylov subspace methods. We accomplish this through a set of computed examples. We keep technical details to a minimum and provide references to the literature. This allows us to demonstrate the mathematical beauty and intricacies of the methods, and to recall some persistent misunderstandings as well as important open problems. We hope that this work can initiate further theoretical investigations of Krylov subspace methods. This paper can not cover all Krylov subspace methods. The principles discussed for CG and GMRES are, however, important for all of them. Further, practical computations always incorporate preconditioning. We will not deal with preconditioning techniques, but we will deal with the basic question of how preconditioning is motivated, and we will recall some recent analytic results.
翻译:本文标题受Forsythe于1952年撰写、1953年发表于《美国数学会通报》第59卷第299-329页的论文标题启发。Forsythe指出求解含$n$个未知数的$n$元线性代数方程组在数学上是一个低层次课题。其优美论述最终归结为当时“最新入选的算法——共轭梯度法”。我们认为,在70年后的今天,重新审视Forsythe的观点以及Hestenes、Stiefel、Lanczos、Karush和Hayes等同期相关研究中的见解是否仍具现实意义至关重要。除共轭梯度法(CG)外,本文还涵盖广义最小残差法(GMRES),并指出我们认为对当前Krylov子空间方法的数学与计算理解至关重要的构成要素。我们通过一组计算实例来实现这一目标,将技术细节控制在最低限度并提供文献参考。这使我们能够展现这些方法的数学之美与复杂性,同时回顾一些长期存在的误解及重要的未解决问题。我们期望本文能推动Krylov子空间方法的进一步理论研究。本文无法涵盖所有Krylov子空间方法,但针对CG和GMRES所讨论的原理对全部此类方法均具重要性。此外,实际计算中始终涉及预处理技术。我们虽不讨论预处理方法,但将阐述预处理的基本动机,并回顾一些近期分析结果。