We study the mixing time of the projected Langevin algorithm (LA) and the privacy curve of noisy Stochastic Gradient Descent (SGD), beyond nonexpansive iterations. Specifically, we derive new mixing time bounds for the projected LA which are, in some important cases, dimension-free and poly-logarithmic on the accuracy, closely matching the existing results in the smooth convex case. Additionally, we establish new upper bounds for the privacy curve of the subsampled noisy SGD algorithm. These bounds show a crucial dependency on the regularity of gradients, and are useful for a wide range of convex losses beyond the smooth case. Our analysis relies on a suitable extension of the Privacy Amplification by Iteration (PABI) framework (Feldman et al., 2018; Altschuler and Talwar, 2022, 2023) to noisy iterations whose gradient map is not necessarily nonexpansive. This extension is achieved by designing an optimization problem which accounts for the best possible Rényi divergence bound obtained by an application of PABI, where the tractability of the problem is crucially related to the modulus of continuity of the associated gradient mapping. We show that, in several interesting cases -- namely the nonsmooth convex, weakly smooth and (strongly) dissipative -- such optimization problem can be solved exactly and explicitly, yielding the tightest possible PABI-based bounds.

翻译：我们研究了投影朗之万算法（LA）的混合时间以及带噪声随机梯度下降（SGD）的隐私曲线，突破了非扩张迭代的限制。具体而言，我们推导了投影朗之万算法新的混合时间上界，这些上界在某些重要情形下与维度无关且对精度呈多对数依赖，与光滑凸情形下的现有结果高度吻合。此外，我们为子采样带噪声SGD算法建立了隐私曲线的新上界。这些上界揭示了其对梯度正则性的关键依赖关系，适用于光滑情形之外更广泛的凸损失函数。我们的分析依赖于对迭代隐私放大（PABI）框架（Feldman等人，2018；Altschuler与Talwar，2022，2023）的适当扩展，使其适用于梯度映射未必非扩张的带噪声迭代。该扩展通过设计一个优化问题实现，该问题旨在通过应用PABI获得最优的Rényi散度界，而问题的可解性关键取决于相关梯度映射的模连续性。我们证明，在若干重要情形——即非光滑凸、弱光滑及（强）耗散情形——此类优化问题可被精确且显式求解，从而得到基于PABI的最紧致界。