Since the development of the conjugate gradient (CG) method in 1952 by Hestenes and Stiefel, CG, has become an indispensable tool in computational mathematics for solving positive definite linear systems. On the other hand, the conjugate residual (CR) method, closely related CG and introduced by Stiefel in 1955 for the same settings, remains relatively less known outside the numerical linear algebra community. Since their inception, these methods -- henceforth collectively referred to as conjugate direction methods -- have been extended beyond positive definite to indefinite, albeit consistent, settings. Going one step further, in this paper, we investigate the theoretical and empirical properties of these methods under inconsistent systems. Among other things, we show that small modifications to the original algorithms allow for the pseudo-inverse solution. Furthermore, we show that CR is essentially equivalent to the minimum residual method, proposed by Paige and Saunders in 1975, in such contexts. Lastly, we conduct a series of numerical experiments to shed lights on their numerical stability (or lack thereof) and their performance for inconsistent systems. Surprisingly, we will demonstrate that, unlike CR and contrary to popular belief, CG can exhibit significant numerical instability, bordering on catastrophe in some instances.

翻译：自Hestenes和Stiefel于1952年提出共轭梯度法以来，CG已成为计算数学中求解正定线性系统不可或缺的工具。另一方面，由Stiefel于1955年在相同背景下提出的共轭残差法，虽与CG密切相关，但在数值线性代数领域之外仍相对鲜为人知。自诞生以来，这些方法——下文统称为共轭方向法——已从正定系统推广至不定但相容的系统。更进一步地，本文研究了这些方法在不一致系统中的理论与实证性质。我们特别证明，对原始算法进行微小修改即可获得伪逆解。此外，我们证明了在此类情境下，CR本质上等价于Paige和Saunders于1975年提出的最小残差法。最后，我们通过一系列数值实验揭示了它们在不一致系统中的数值稳定性（或不稳定性）及性能表现。令人惊讶的是，我们将证明与CR不同且与普遍认知相反，CG可能表现出显著的数值不稳定性，在某些情况下甚至接近灾难性失效。