Differentially private (DP) machine learning algorithms incur many sources of randomness, such as random initialization, random batch subsampling, and shuffling. However, such randomness is difficult to take into account when proving differential privacy bounds because it induces mixture distributions for the algorithm's output that are difficult to analyze. This paper focuses on improving privacy bounds for shuffling models and one-iteration differentially private gradient descent (DP-GD) with random initializations using $f$-DP. We derive a closed-form expression of the trade-off function for shuffling models that outperforms the most up-to-date results based on $(\epsilon,\delta)$-DP. Moreover, we investigate the effects of random initialization on the privacy of one-iteration DP-GD. Our numerical computations of the trade-off function indicate that random initialization can enhance the privacy of DP-GD. Our analysis of $f$-DP guarantees for these mixture mechanisms relies on an inequality for trade-off functions introduced in this paper. This inequality implies the joint convexity of $F$-divergences. Finally, we study an $f$-DP analog of the advanced joint convexity of the hockey-stick divergence related to $(\epsilon,\delta)$-DP and apply it to analyze the privacy of mixture mechanisms.

翻译：差分隐私机器学习算法涉及多种随机性来源，例如随机初始化、随机批次子采样和洗牌。然而，在证明差分隐私界时，这类随机性难以被纳入考量，因为它会导致算法输出产生难以分析的混合分布。本文聚焦于利用$f$-差分隐私改进洗牌模型和含随机初始化的单次迭代差分隐私梯度下降算法的隐私界。我们推导出洗牌模型权衡函数的闭式表达式，该表达式超越了基于$(\epsilon,\delta)$-差分隐私的最新结果。进一步，我们探究了随机初始化对单次迭代DP-GD隐私性的影响。权衡函数的数值计算表明，随机初始化能增强DP-GD的隐私保障。我们对这些混合机制的$f$-差分隐私分析依赖于本文引入的一个权衡函数不等式，该不等式蕴含了$F\)-散度的联合凸性。最后，我们研究了与$(\epsilon,\delta)$-差分隐私相关的冰球棒散度高级联合凸性的$f$-差分隐私类比，并将其应用于分析混合机制的隐私性。