Sampling from a target distribution induced by training data is central to Bayesian learning, with Stochastic Gradient Langevin Dynamics (SGLD) serving as a key tool for scalable posterior sampling and decentralized variants enabling learning when data are distributed across a network of agents. This paper introduces DIGing-SGLD, a decentralized SGLD algorithm designed for scalable Bayesian learning in multi-agent systems operating over time-varying networks. Existing decentralized SGLD methods are restricted to static network topologies, and many exhibit steady-state sampling bias caused by network effects, even when full batches are used. DIGing-SGLD overcomes these limitations by integrating Langevin-based sampling with the gradient-tracking mechanism of the DIGing algorithm, originally developed for decentralized optimization over time-varying networks, thereby enabling efficient and bias-free sampling without a central coordinator. To our knowledge, we provide the first finite-time non-asymptotic Wasserstein convergence guarantees for decentralized SGLD-based sampling over time-varying networks, with explicit constants. Under standard strong convexity and smoothness assumptions, DIGing-SGLD achieves geometric convergence to an $O(\sqrtη)$ neighborhood of the target distribution, where $η$ is the stepsize, with dependence on the target accuracy matching the best-known rates for centralized and static-network SGLD algorithms using constant stepsize. Numerical experiments on Bayesian linear and logistic regression validate the theoretical results and demonstrate the strong empirical performance of DIGing-SGLD under dynamically evolving network conditions.

翻译：从训练数据诱导的目标分布中采样是贝叶斯学习的核心，其中随机梯度朗之万动力学（SGLD）作为可扩展后验采样的关键工具，而其去中心化变体使得数据分布在多智能体网络时能够实现学习。本文提出DIGing-SGLD，一种专为在时变网络上运行的多智能体系统中进行可扩展贝叶斯学习而设计的去中心化SGLD算法。现有的去中心化SGLD方法局限于静态网络拓扑，且许多方法即使使用全批次数据，也会因网络效应产生稳态采样偏差。DIGing-SGLD通过将基于朗之万的采样与DIGing算法的梯度跟踪机制相结合，克服了这些限制——DIGing算法最初为时变网络上的去中心化优化而开发，从而实现了无需中央协调器的高效且无偏采样。据我们所知，我们首次为时变网络上基于去中心化SGLD的采样提供了具有显式常数的有限时间非渐近Wasserstein收敛保证。在标准的强凸性和光滑性假设下，DIGing-SGLD以几何速率收敛到目标分布的$O(\sqrtη)$邻域，其中$η$为步长，其对目标精度的依赖性与使用恒定步长的集中式和静态网络SGLD算法的最佳已知速率相匹配。在贝叶斯线性回归和逻辑回归上的数值实验验证了理论结果，并展示了DIGing-SGLD在动态演化网络条件下的强大实证性能。