The Stochastic Gradient Langevin Dynamics (SGLD) are popularly used to approximate Bayesian posterior distributions in statistical learning procedures with large-scale data. As opposed to many usual Markov chain Monte Carlo (MCMC) algorithms, SGLD is not stationary with respect to the posterior distribution; two sources of error appear: The first error is introduced by an Euler--Maruyama discretisation of a Langevin diffusion process, the second error comes from the data subsampling that enables its use in large-scale data settings. In this work, we consider an idealised version of SGLD to analyse the method's pure subsampling error that we then see as a best-case error for diffusion-based subsampling MCMC methods. Indeed, we introduce and study the Stochastic Gradient Langevin Diffusion (SGLDiff), a continuous-time Markov process that follows the Langevin diffusion corresponding to a data subset and switches this data subset after exponential waiting times. There, we show that the Wasserstein distance between the posterior and the limiting distribution of SGLDiff is bounded above by a fractional power of the mean waiting time. Importantly, this fractional power does not depend on the dimension of the state space. We bring our results into context with other analyses of SGLD.

翻译：随机梯度Langevin动力学（SGLD）被广泛用于大规模数据统计学习过程中近似贝叶斯后验分布。与许多常见的马尔可夫链蒙特卡洛（MCMC）算法不同，SGLD相对于后验分布并非平稳；其中存在两种误差来源：第一种误差来自Langevin扩散过程的欧拉-丸山离散化，第二种误差来自数据子采样，这使其能够应用于大规模数据场景。本研究考虑SGLD的理想化版本，以分析该方法中纯粹的子采样误差，并将其视为基于扩散的子采样MCMC方法的最优情形误差。具体而言，我们引入并研究了随机梯度Langevin扩散（SGLDiff），这是一种连续时间马尔可夫过程，它遵循对应数据子集的Langevin扩散，并在指数等待时间后切换该数据子集。我们证明，后验分布与SGLDiff极限分布之间的Wasserstein距离受平均等待时间的分数幂上界约束。重要的是，该分数幂不依赖于状态空间的维度。我们将结果与SGLD的其他分析进行比较。