Synthetic data has been hailed as the silver bullet for privacy preserving data analysis. If a record is not real, then how could it violate a person's privacy? In addition, deep-learning based generative models are employed successfully to approximate complex high-dimensional distributions from data and draw realistic samples from this learned distribution. It is often overlooked though that generative models are prone to memorising many details of individual training records and often generate synthetic data that too closely resembles the underlying sensitive training data, hence violating strong privacy regulations as, e.g., encountered in health care. Differential privacy is the well-known state-of-the-art framework for guaranteeing protection of sensitive individuals' data, allowing aggregate statistics and even machine learning models to be released publicly without compromising privacy. The training mechanisms however often add too much noise during the training process, and thus severely compromise the utility of these private models. Even worse, the tight privacy budgets do not allow for many training epochs so that model quality cannot be properly controlled in practice. In this paper we explore an alternative approach for privately generating data that makes direct use of the inherent stochasticity in generative models, e.g., variational autoencoders. The main idea is to appropriately constrain the continuity modulus of the deep models instead of adding another noise mechanism on top. For this approach, we derive mathematically rigorous privacy guarantees and illustrate its effectiveness with practical experiments.

翻译：合成数据被誉为隐私保护数据分析的银弹——如果一条记录并非真实存在，又怎能侵犯个人隐私呢？此外，基于深度学习的生成模型被成功用于从数据中逼近复杂的高维分布，并从该学习分布中抽取逼真的样本。然而，人们常常忽略，生成模型容易记忆大量个体训练记录的细节，并经常生成与底层敏感训练数据高度相似的合成数据，从而违反了例如医疗领域所遇到的严格隐私法规。差分隐私是保障敏感个体数据保护的知名先进框架，它允许公开发布聚合统计量乃至机器学习模型而不损害隐私。然而，其训练机制往往在训练过程中添加过多噪声，严重损害了这些私有模型的实用性。更糟的是，严格的隐私预算不允许过多的训练轮次，导致实践中无法有效控制模型质量。本文探索了一种隐私数据生成的替代方法，该方法直接利用生成模型（如变分自编码器）固有的随机性。其主要思想是适当约束深度模型的连续性模量，而非在模型之上额外添加噪声机制。针对这一方法，我们推导了数学上严格的隐私保证，并通过实际实验验证了其有效性。