A deep learning model usually has to sacrifice some utilities when it acquires some other abilities or characteristics. Privacy preservation has such trade-off relationships with utilities. The loss disparity between various defense approaches implies the potential to decouple generalizability and privacy risks to maximize privacy gain. In this paper, we identify that the model's generalization and privacy risks exist in different regions in deep neural network architectures. Based on the observations that we investigate, we propose Privacy-Preserving Training Principle (PPTP) to protect model components from privacy risks while minimizing the loss in generalizability. Through extensive evaluations, our approach shows significantly better maintenance in model generalizability while enhancing privacy preservation.

翻译：深度学习模型在获得某些能力或特性时通常需要牺牲部分效用。隐私保护与模型效用之间也存在此类权衡关系。不同防御方法之间的损失差异表明，存在通过解耦泛化能力与隐私风险来最大化隐私增益的潜在可能。本文发现，在深度神经网络架构中，模型的泛化风险与隐私风险存在于不同区域。基于所考察的观测结果，我们提出隐私保护训练原则（PPTP），在最小化泛化能力损失的同时保护模型组件免受隐私风险。通过广泛评估，本方法在增强隐私保护的同时，展现出显著更优的模型泛化能力保持效果。