Probabilistic linear solvers (PLSs) return probability distributions that quantify uncertainty due to limited computation in the solution of linear systems. The literature has traditionally distinguished between Bayesian PLSs, which condition a prior on information obtained from projections of the linear system, and probabilistic iterative methods (PIMs), which lift classical iterative solvers to probability space. In this work we show this dichotomy to be false: Bayesian PLSs are a special case of non-stationary affine PIMs. In addition, we prove that any realistic affine PIM is calibrated. These results motivate a focus on (non-stationary) affine PIMs, but their practical adoption has been limited by the significant manual effort required to implement them. To address this, we introduce affine tracing, an algorithmic framework that automatically constructs a PIM from a standard implementation of an affine iterative method by passing symbolic tracers through the computation to build an affine computational graph. We show how this graph can be transformed to compute posterior covariances, and how equality saturation can be used to perform algebraic simplifications required for computation under specific prior choices. We demonstrate the framework by automatically generating a probabilistic multigrid solver and evaluate its performance in the context of Gaussian process approximation.
翻译:概率线性求解器(PLS)返回概率分布,用于量化线性系统求解过程中因有限计算而产生的不确定性。传统文献将贝叶斯PLS(基于线性系统投影信息对先验进行条件化)与概率迭代方法(PIM,将经典迭代求解器提升至概率空间)区分为两类。本文证明这一二分法是错误的:贝叶斯PLS是非平稳仿射PIM的特例。此外,我们证明任何实际仿射PIM都是校准的。这些结论促使学界聚焦(非平稳)仿射PIM,但其实际应用受限于实现所需的大量人工操作。为此,我们提出仿射追踪——一种通过向计算过程传递符号追踪器以构建仿射计算图,从而从仿射迭代方法的标准实现中自动构建PIM的算法框架。本文展示了如何将该计算图转化为后验协方差计算,以及如何利用等式饱和技术在特定先验选择下执行所需的代数简化。我们通过自动生成概率多重网格求解器验证该框架,并在高斯过程近似场景中评估其性能。