Differential privacy (DP) has become an essential framework for privacy-preserving machine learning. Existing DP learning methods, however, often have disparate impacts on model predictions, e.g., for minority groups. Gradient clipping, which is often used in DP learning, can suppress larger gradients from challenging samples. We show that this problem is amplified by adaptive clipping, which will often shrink the clipping bound to tiny values to match a well-fitting majority, while significantly reducing the accuracy for others. We propose bounded adaptive clipping, which introduces a tunable lower bound to prevent excessive gradient suppression. Our method improves worst-class accuracy by over 10 percentage points on Skewed and Fashion MNIST compared to unbounded adaptive clipping, 7 points compared to Automatic clipping, and 5 points compared to constant clipping. The code is available at https://github.com/TrustworthyMLHelsinki/adaptive-clipping-fairness.

翻译：差分隐私已成为隐私保护机器学习的基本框架。然而，现有的差分隐私学习方法往往对模型预测产生差异性影响，例如对少数群体而言。梯度裁剪是差分隐私学习中的常用技术，但可能会抑制困难样本产生的较大梯度。我们证明，自适应裁剪会加剧这一问题：裁剪界限往往被缩小至极小值以拟合拟合良好的多数群体，同时显著降低其他群体的准确率。为此，我们提出有界自适应裁剪，通过引入可调节的下界来防止过度梯度抑制。与无界自适应裁剪相比，我们的方法在Skewed和Fashion MNIST数据集上的最差类别准确率提升超过10个百分点；相较自动裁剪提升7个百分点；相较固定裁剪提升5个百分点。代码开源于https://github.com/TrustworthyMLHelsinki/adaptive-clipping-fairness。