Machine learning models are susceptible to membership inference attacks (MIAs), which aim to infer whether a sample is in the training set. Existing work utilizes gradient ascent to enlarge the loss variance of training data, alleviating the privacy risk. However, optimizing toward a reverse direction may cause the model parameters to oscillate near local minima, leading to instability and suboptimal performance. In this work, we propose a novel method -- Convex-Concave Loss, which enables a high variance of training loss distribution by gradient descent. Our method is motivated by the theoretical analysis that convex losses tend to decrease the loss variance during training. Thus, our key idea behind CCL is to reduce the convexity of loss functions with a concave term. Trained with CCL, neural networks produce losses with high variance for training data, reinforcing the defense against MIAs. Extensive experiments demonstrate the superiority of CCL, achieving state-of-the-art balance in the privacy-utility trade-off.

翻译：机器学习模型易受成员推断攻击（MIAs），该攻击旨在推断样本是否属于训练集。现有研究采用梯度上升法增大训练数据的损失方差，以缓解隐私风险。然而，向反方向优化可能导致模型参数在局部最小值附近振荡，引发不稳定性及次优性能。本文提出一种新方法——凸-凹损失（Convex-Concave Loss, CCL），该方法通过梯度下降实现训练损失分布的高方差。我们的方法基于理论分析：凸损失倾向于在训练过程中减小损失方差。因此，CCL的核心思想是通过引入凹项降低损失函数的凸性。采用CCL训练的神经网络能产生高方差的训练数据损失，从而强化对MIAs的防御能力。大量实验证明了CCL的优越性，其在隐私-效用权衡中达到了当前最优平衡。