Bayesian learning via Stochastic Gradient Langevin Dynamics (SGLD) has been suggested for differentially private learning. While previous research provides differential privacy bounds for SGLD at the initial steps of the algorithm or when close to convergence, the question of what differential privacy guarantees can be made in between remains unanswered. This interim region is of great importance, especially for Bayesian neural networks, as it is hard to guarantee convergence to the posterior. This paper shows that using SGLD might result in unbounded privacy loss for this interim region, even when sampling from the posterior is as differentially private as desired.

翻译：通过随机梯度朗之万动力学（SGLD）进行贝叶斯学习已被提议用于差分隐私学习。尽管先前的研究在算法初始步骤或接近收敛时提供了SGLD的差分隐私界限，但在此中间阶段能提供何种差分隐私保证的问题仍未得到解答。这个中间区域非常重要，尤其对于贝叶斯神经网络而言，因为很难保证收敛到后验分布。本文表明，即使从后验分布采样的差分隐私性可以达到任意理想程度，使用SGLD也可能导致该中间区域出现无界的隐私损失。