Machine learning models are susceptible to membership inference attacks (MIAs), which aim to infer whether a sample is in the training set. Existing work utilizes gradient ascent to enlarge the loss variance of training data, alleviating the privacy risk. However, optimizing toward a reverse direction may cause the model parameters to oscillate near local minima, leading to instability and suboptimal performance. In this work, we propose a novel method -- Convex-Concave Loss, which enables a high variance of training loss distribution by gradient descent. Our method is motivated by the theoretical analysis that convex losses tend to decrease the loss variance during training. Thus, our key idea behind CCL is to reduce the convexity of loss functions with a concave term. Trained with CCL, neural networks produce losses with high variance for training data, reinforcing the defense against MIAs. Extensive experiments demonstrate the superiority of CCL, achieving state-of-the-art balance in the privacy-utility trade-off.

翻译：机器学习模型易受成员推理攻击的影响，此类攻击旨在推断样本是否属于训练集。现有研究利用梯度上升扩大训练数据的损失方差，以减轻隐私风险。然而，逆向优化可能导致模型参数在局部极小值附近振荡，引发不稳定性和次优性能。本文提出一种新方法——凸凹损失，该方法通过梯度下降实现训练损失分布的高方差。我们的方法基于以下理论分析：凸损失在训练过程中倾向于降低损失方差。因此，CCL的核心思想是通过引入凹项来降低损失函数的凸性。使用CCL训练的神经网络能为训练数据生成高方差的损失，从而增强对成员推理攻击的防御能力。大量实验证明CCL的优越性，其在隐私-效用权衡中实现了最先进的平衡。