We propose a federated averaging Langevin algorithm (FA-LD) for uncertainty quantification and mean predictions with distributed clients. In particular, we generalize beyond normal posterior distributions and consider a general class of models. We develop theoretical guarantees for FA-LD for strongly log-concave distributions with non-i.i.d data and study how the injected noise and the stochastic-gradient noise, the heterogeneity of data, and the varying learning rates affect the convergence. Such an analysis sheds light on the optimal choice of local updates to minimize communication costs. Important to our approach is that the communication efficiency does not deteriorate with the injected noise in the Langevin algorithms. In addition, we examine in our FA-LD algorithm both independent and correlated noise used over different clients. We observe there is a trade-off between the pairs among communication, accuracy, and data privacy. As local devices may become inactive in federated networks, we also show convergence results based on different averaging schemes where only partial device updates are available. In such a case, we discover an additional bias that does not decay to zero.

翻译：我们提出了一种联邦平均朗之万算法（FA-LD），用于分布式客户端的 uncertainty 量化与均值预测。特别地，我们将研究范围从正态后验分布推广至更一般的模型类别。针对非独立同分布数据下的强对数凹分布，我们建立了FA-LD的理论保证，并探讨了注入噪声、随机梯度噪声、数据异质性以及变学习率对收敛性的影响。此类分析揭示了为最小化通信成本而选择局部更新最优策略的规律。我们的方法中一个关键特性是：通信效率不会因朗之万算法中的注入噪声而恶化。此外，我们在FA-LD算法中考察了不同客户端上使用的独立噪声与相关噪声。我们发现通信效率、精度与数据隐私三者之间存在权衡关系。考虑到联邦网络中本地设备可能处于非活跃状态，我们还展示了基于不同平均方案（仅包含部分设备更新）的收敛性结果。在此情况下，我们发现了不会衰减至零的额外偏差。