Machine Learning (ML) is accelerating progress across fields and industries, but relies on accessible and high-quality training data. Some of the most important datasets are found in biomedical and financial domains in the form of spreadsheets and relational databases. But this tabular data is often sensitive in nature. Synthetic data generation offers the potential to unlock sensitive data, but generative models tend to memorise and regurgitate training data, which undermines the privacy goal. To remedy this, researchers have incorporated the mathematical framework of Differential Privacy (DP) into the training process of deep neural networks. But this creates a trade-off between the quality and privacy of the resulting data. Generative Adversarial Networks (GANs) are the dominant paradigm for synthesising tabular data under DP, but suffer from unstable adversarial training and mode collapse, which are exacerbated by the privacy constraints and challenging tabular data modality. This work optimises the quality-privacy trade-off of generative models, producing higher quality tabular datasets with the same privacy guarantees. We implement novel end-to-end models that leverage attention mechanisms to learn reversible tabular representations. We also introduce TableDiffusion, the first differentially-private diffusion model for tabular data synthesis. Our experiments show that TableDiffusion produces higher-fidelity synthetic datasets, avoids the mode collapse problem, and achieves state-of-the-art performance on privatised tabular data synthesis. By implementing TableDiffusion to predict the added noise, we enabled it to bypass the challenges of reconstructing mixed-type tabular data. Overall, the diffusion paradigm proves vastly more data and privacy efficient than the adversarial paradigm, due to augmented re-use of each data batch and a smoother iterative training process.

翻译：机器学习正加速各领域与行业的进步，但其依赖高质量且易于获取的训练数据。最具价值的数据库常以电子表格和关系数据库形式存在于生物医学与金融领域，然而这类表格数据本质上具有敏感性。合成数据生成技术有望释放敏感数据价值，但生成模型容易记忆并复现训练数据，这违背了隐私保护目标。为此，研究人员将差分隐私（DP）的数学框架融入深度神经网络训练过程，但这会导致生成数据的质量与隐私性之间的权衡。生成对抗网络（GANs）是实现差分隐私表格数据合成的主流范式，但其面临对抗训练不稳定和模式崩塌问题，而隐私约束与表格数据模态的复杂性进一步加剧了这些挑战。本研究通过优化生成模型的质量-隐私权衡，在相同隐私保障下生成更高质量的表格数据集。我们实现了利用注意力机制学习可逆表格表示的新型端到端模型，并首次提出用于表格数据合成的差分隐私扩散模型TableDiffusion。实验表明，TableDiffusion能生成保真度更高的合成数据集，有效避免模式崩塌问题，在隐私化表格数据合成领域达到当前最优性能。通过让TableDiffusion直接预测添加噪声量，我们使其绕过了混合类型表格数据重构的难题。总体而言，扩散范式因增强性数据批次重复利用和平滑的迭代训练过程，在数据效率和隐私效率上均显著优于对抗范式。