Mechanisms used in privacy-preserving machine learning often aim to guarantee differential privacy (DP) during model training. Practical DP-ensuring training methods use randomization when fitting model parameters to privacy-sensitive data (e.g., adding Gaussian noise to clipped gradients). We demonstrate that such randomization incurs predictive multiplicity: for a given input example, the output predicted by equally-private models depends on the randomness used in training. Thus, for a given input, the predicted output can vary drastically if a model is re-trained, even if the same training dataset is used. The predictive-multiplicity cost of DP training has not been studied, and is currently neither audited for nor communicated to model designers and stakeholders. We derive a bound on the number of re-trainings required to estimate predictive multiplicity reliably. We analyze--both theoretically and through extensive experiments--the predictive-multiplicity cost of three DP-ensuring algorithms: output perturbation, objective perturbation, and DP-SGD. We demonstrate that the degree of predictive multiplicity rises as the level of privacy increases, and is unevenly distributed across individuals and demographic groups in the data. Because randomness used to ensure DP during training explains predictions for some examples, our results highlight a fundamental challenge to the justifiability of decisions supported by differentially private models in high-stakes settings. We conclude that practitioners should audit the predictive multiplicity of their DP-ensuring algorithms before deploying them in applications of individual-level consequence.

翻译：隐私保护机器学习中常用的机制旨在确保模型训练过程中的差分隐私。实际应用差分隐私的保障性训练方法在拟合模型参数到隐私敏感数据时采用随机化技术（例如向裁剪后的梯度添加高斯噪声）。我们证明这种随机化会导致预测的多重性：对于给定的输入样本，具有相同隐私保护等级的模型输出的预测结果取决于训练过程中使用的随机性。因此，即使使用相同训练数据集重新训练模型，特定输入的预测输出也可能发生剧烈变化。差分隐私训练的预测多重性成本尚未得到研究，目前既未被审计，也未向模型设计者和利益相关者传达。我们推导出可靠估计预测多重性所需的重新训练次数界限。通过理论分析与大量实验，我们研究了三种保障差分隐私的算法——输出扰动、目标扰动和DP-SGD——的预测多重性成本。结果表明，随着隐私保护水平的提升，预测多重性程度相应增加，且这种多重性在数据集的个体和人口统计群体间分布不均。由于训练过程中用于保证差分隐私的随机性解释部分样本的预测结果，我们的研究揭示了在高风险场景中基于差分隐私模型的决策可辩解性面临的根本挑战。我们建议实践者在部署影响个体利益的应用程序前，应审计其差分隐私保障算法的预测多重性。