Privacy amplification exploits randomness in data selection to provide tighter differential privacy (DP) guarantees. This analysis is key to DP-SGD's success in machine learning, but, is not readily applicable to the newer state-of-the-art algorithms. This is because these algorithms, known as DP-FTRL, use the matrix mechanism to add correlated noise instead of independent noise as in DP-SGD. In this paper, we propose "MMCC", the first algorithm to analyze privacy amplification via sampling for any generic matrix mechanism. MMCC is nearly tight in that it approaches a lower bound as $\epsilon\to0$. To analyze correlated outputs in MMCC, we prove that they can be analyzed as if they were independent, by conditioning them on prior outputs. Our "conditional composition theorem" has broad utility: we use it to show that the noise added to binary-tree-DP-FTRL can asymptotically match the noise added to DP-SGD with amplification. Our amplification algorithm also has practical empirical utility: we show it leads to significant improvement in the privacy-utility trade-offs for DP-FTRL algorithms on standard benchmarks.

翻译：隐私放大利用数据选择中的随机性来提供更紧的差分隐私保证。这一分析是DP-SGD在机器学习中成功的关键，但尚不适用于最新的先进算法。这是因为这些被称为DP-FTRL的算法使用矩阵机制添加相关噪声，而非DP-SGD中的独立噪声。在本文中，我们提出"MMCC"，这是首个分析任意通用矩阵机制通过采样实现隐私放大的算法。MMCC几乎是紧致的，当ϵ→0时趋近于下界。为分析MMCC中的相关输出，我们证明可通过将先前输出作为条件，将其视为独立输出进行分析。我们的"条件组合定理"具有广泛适用性：我们利用该定理证明，二元树DP-FTRL添加的噪声在渐近意义上可与具有放大的DP-SGD添加的噪声相匹配。我们的放大算法还具有实践层面的实证效用：我们证明，在标准基准测试中，该算法能够显著提升DP-FTRL算法的隐私-效用权衡。