Gaussian processes (GPs) based methods for solving partial differential equations (PDEs) demonstrate great promise by bridging the gap between the theoretical rigor of traditional numerical algorithms and the flexible design of machine learning solvers. The main bottleneck of GP methods lies in the inversion of a covariance matrix, whose cost grows cubically concerning the size of samples. Drawing inspiration from neural networks, we propose a mini-batch algorithm combined with GPs to solve nonlinear PDEs. A naive deployment of a stochastic gradient descent method for solving PDEs with GPs is challenging, as the objective function in the requisite minimization problem cannot be depicted as the expectation of a finite-dimensional random function. To address this issue, we employ a mini-batch method to the corresponding infinite-dimensional minimization problem over function spaces. The algorithm takes a mini-batch of samples at each step to update the GP model. Thus, the computational cost is allotted to each iteration. Using stability analysis and convexity arguments, we show that the mini-batch method steadily reduces a natural measure of errors towards zero at the rate of $O(1/K+1/M)$, where $K$ is the number of iterations and $M$ is the batch size.

翻译：基于高斯过程（GPs）的偏微分方程（PDEs）求解方法通过弥合传统数值算法的理论严谨性与机器学习求解器的灵活设计之间的差距，展现出巨大潜力。GP方法的主要瓶颈在于协方差矩阵的求逆运算，其计算成本随样本量呈三次方增长。受神经网络启发，我们提出一种结合高斯过程的小批量算法以求解非线性PDEs。由于所需求解极小化问题中的目标函数无法表示为有限维随机函数的期望，直接采用随机梯度下降法求解高斯过程PDEs面临困难。为解决该问题，我们将小批量方法应用于函数空间上的相应无穷维极小化问题。该算法在每步迭代中采用小批量样本更新GP模型，从而将计算成本分摊至各次迭代。通过稳定性分析与凸性论证，我们证明该小批量方法能以$O(1/K+1/M)$的速率稳定降低误差的自然度量，其中$K$为迭代次数，$M$为批量大小。