Differentially private (DP) image synthesis generates images that preserve the statistical characteristics of a sensitive dataset, enabling sensitive data analysis and usage while providing rigorous guarantees of privacy leakage. Existing methods fine-tune public models using DP Stochastic Gradient Descent (DP-SGD) on sensitive images to generate synthetic images. But full fine-tuning public models on sensitive images is computationally expensive, because current public models typically contain a large number of parameters. Recent work proposes heuristically using Low-Rank Adaptation (LoRA) on all attention-layer parameters of public models to reduce the number of trainable parameters. However, we argue that exhaustive LoRA coverage across all attention-layer parameters is suboptimal in a DP setting, as it leads to noise accumulation and collapse during private training. To address this issue, we propose DP-SAPF, which uses a saliency-aware strategy to identify specific target parameters for LoRA training under DP. DP-SAPF is inspired by the fact that larger gradients signify higher saliency, indicating that these parameters are most critical for the DP learning. Specifically, we feed the sensitive images into public models, compute gradients, and add noise to the gradients to satisfy DP. Then, DP-SAPF identifies the most salient parameters, those exhibiting high gradient magnitudes on sensitive images, for DP fine-tuning. Experiments on four sensitive image datasets show that DP-SAPF improves the utility and fidelity of synthetic images while requiring fewer computational resources than fine-tuning methods without parameter selection.

翻译：差分隐私（DP）图像合成能够生成保留敏感数据集统计特征的图像，在提供严格隐私泄露保证的同时，支持敏感数据的分析与使用。现有方法在敏感图像上使用差分隐私随机梯度下降（DP-SGD）对公共模型进行微调，以生成合成图像。然而，由于当前公共模型通常包含大量参数，在敏感图像上完全微调公共模型计算成本高昂。近期研究启发式地提出对公共模型所有注意力层参数采用低秩适配（LoRA），以减少可训练参数数量。但我们认为，在差分隐私场景下，对所有注意力层参数实施全覆盖LoRA并非最优策略，因为这会导致私有训练过程中噪声累积与性能崩塌。为解决此问题，我们提出DP-SAPF，该方法利用显著性感知策略识别特定的目标参数，在差分隐私约束下进行LoRA训练。DP-SAPF的灵感源于：梯度幅值越大表示参数显著性越高，表明这些参数对差分隐私学习最为关键。具体而言，我们将敏感图像输入公共模型，计算梯度，并对梯度添加噪声以满足差分隐私要求。随后，DP-SAPF识别出对敏感图像梯度幅值最高的最显著参数，进行差分隐私微调。在四个敏感图像数据集上的实验表明，与未进行参数选择的微调方法相比，DP-SAPF在提升合成图像效用与保真度的同时，所需计算资源更少。