Storage-efficient privacy-preserving learning is crucial due to increasing amounts of sensitive user data required for modern learning tasks. We propose a framework for reducing the storage cost of user data while at the same time providing privacy guarantees, without essential loss in the utility of the data for learning. Our method comprises noise injection followed by lossy compression. We show that, when appropriately matching the lossy compression to the distribution of the added noise, the compressed examples converge, in distribution, to that of the noise-free training data as the sample size of the training data (or the dimension of the training data) increases. In this sense, the utility of the data for learning is essentially maintained, while reducing storage and privacy leakage by quantifiable amounts. We present experimental results on the CelebA dataset for gender classification and find that our suggested pipeline delivers in practice on the promise of the theory: the individuals in the images are unrecognizable (or less recognizable, depending on the noise level), overall storage of the data is substantially reduced, with no essential loss (and in some cases a slight boost) to the classification accuracy. As an added bonus, our experiments suggest that our method yields a substantial boost to robustness in the face of adversarial test data.

翻译：存储高效的隐私保护学习对于现代学习任务中日益增长的敏感用户数据至关重要。我们提出了一种框架，在降低用户数据存储成本的同时提供隐私保障，且不损失数据对学习任务的实用性。该方法包含噪声注入与有损压缩两个步骤。研究表明，当有损压缩策略与注入噪声的分布相匹配时，压缩样本的分布会随着训练数据样本量（或维度）的增加而趋近于无噪声训练数据的分布。在此意义上，数据的学习效用基本得以保持，同时存储空间与隐私泄露风险均可量化降低。我们在CelebA数据集上进行了性别分类实验，结果表明：本文提出的流水线在实践中兑现了理论承诺——图像中的个体变得不可识别（或根据噪声水平降低可识别度），数据总存储量显著减少，而分类精度基本无损失（部分情况下甚至略有提升）。作为附加优势，实验表明该方法能显著增强模型对对抗性测试数据的鲁棒性。