We study the computational scalability of a Gaussian process (GP) framework for solving general nonlinear partial differential equations (PDEs). This framework transforms solving PDEs to solving quadratic optimization problem with nonlinear constraints. Its complexity bottleneck lies in computing with dense kernel matrices obtained from pointwise evaluations of the covariance kernel of the GP and its partial derivatives at collocation points. We present a sparse Cholesky factorization algorithm for such kernel matrices based on the near-sparsity of the Cholesky factor under a new ordering of Diracs and derivative measurements. We rigorously identify the sparsity pattern and quantify the exponentially convergent accuracy of the corresponding Vecchia approximation of the GP, which is optimal in the Kullback-Leibler divergence. This enables us to compute $\epsilon$-approximate inverse Cholesky factors of the kernel matrices with complexity $O(N\log^d(N/\epsilon))$ in space and $O(N\log^{2d}(N/\epsilon))$ in time. With the sparse factors, gradient-based optimization methods become scalable. Furthermore, we can use the oftentimes more efficient Gauss-Newton method, for which we apply the conjugate gradient algorithm with the sparse factor of a reduced kernel matrix as a preconditioner to solve the linear system. We numerically illustrate our algorithm's near-linear space/time complexity for a broad class of nonlinear PDEs such as the nonlinear elliptic, Burgers, and Monge-Amp\`ere equations. In summary, we provide a fast, scalable, and accurate method for solving general PDEs with GPs.

翻译：我们研究了一种基于高斯过程（GP）的框架在求解一般非线性偏微分方程（PDEs）时的计算可扩展性。该框架将PDE求解转化为带非线性约束的二次优化问题。其计算瓶颈在于需要处理由GP协方差核及其偏导数在配置点处逐点评估所得到的稠密核矩阵。我们提出了一种针对此类核矩阵的稀疏Cholesky分解算法，该算法基于在狄拉克δ函数和导数测量的新排序下Cholesky因子的近似稀疏性。我们严格识别了稀疏模式，并量化了对应的GP Vecchia逼近的指数收敛精度——该逼近在Kullback-Leibler散度意义上是最优的。这使得我们能够以空间复杂度$O(N\log^d(N/\epsilon))$和时间复杂度$O(N\log^{2d}(N/\epsilon))$计算核矩阵的$\epsilon$近似逆Cholesky因子。借助稀疏因子，基于梯度的优化方法变得可扩展。此外，我们可以使用通常更高效的Gauss-Newton方法，其中我们应用共轭梯度算法，并将简化核矩阵的稀疏因子作为预处理子来求解线性系统。我们通过数值实验展示了算法在求解非线性椭圆方程、Burgers方程和Monge-Ampère方程等广泛非线性PDEs时具有近线性空间/时间复杂度。总之，我们为基于GP求解一般PDEs提供了一种快速、可扩展且精确的方法。