Differentially private (DP) machine learning algorithms incur many sources of randomness, such as random initialization, random batch subsampling, and shuffling. However, such randomness is difficult to take into account when proving differential privacy bounds because it induces mixture distributions for the algorithm's output that are difficult to analyze. This paper focuses on improving privacy bounds for shuffling models and one-iteration differentially private gradient descent (DP-GD) with random initializations using $f$-DP. We derive a closed-form expression of the trade-off function for shuffling models that outperforms the most up-to-date results based on $(\epsilon,\delta)$-DP. Moreover, we investigate the effects of random initialization on the privacy of one-iteration DP-GD. Our numerical computations of the trade-off function indicate that random initialization can enhance the privacy of DP-GD. Our analysis of $f$-DP guarantees for these mixture mechanisms relies on an inequality for trade-off functions introduced in this paper. This inequality implies the joint convexity of $F$-divergences. Finally, we study an $f$-DP analog of the advanced joint convexity of the hockey-stick divergence related to $(\epsilon,\delta)$-DP and apply it to analyze the privacy of mixture mechanisms.

翻译：差分隐私机器学习算法涉及多种随机性来源，例如随机初始化、随机批次子采样和洗牌。然而，在证明差分隐私界限时，这类随机性难以纳入考量，因其会导致算法输出呈现难以分析的混合分布。本文聚焦于利用 $f$-差分隐私改进洗牌模型及含随机初始化的单次迭代差分隐私梯度下降法的隐私界限。我们推导出洗牌模型的权衡函数闭式表达式，该性能优于基于 $(\epsilon,\delta)$-差分隐私的最新结果。此外，我们研究了随机初始化对单次迭代差分隐私梯度下降隐私性的影响。权衡函数的数值计算表明，随机初始化能增强差分隐私梯度下降的隐私保护能力。本文对这类混合机制的 $f$-差分隐私保证分析，依赖于本文提出的权衡函数不等式。该不等式蕴含 $F$-散度的联合凸性。最后，我们研究了与 $(\epsilon,\delta)$-差分隐私相关的曲棍球棒散度高级联合凸性的 $f$-差分隐私类比，并将其应用于分析混合机制的隐私性。