We propose a new family of label randomizers for training regression models under the constraint of label differential privacy (DP). In particular, we leverage the trade-offs between bias and variance to construct better label randomizers depending on a privately estimated prior distribution over the labels. We demonstrate that these randomizers achieve state-of-the-art privacy-utility trade-offs on several datasets, highlighting the importance of reducing bias when training neural networks with label DP. We also provide theoretical results shedding light on the structural properties of the optimal unbiased randomizers.

翻译：我们提出了一种新的标签随机化器族，用于在标签差分隐私约束下训练回归模型。具体而言，我们利用偏差与方差之间的权衡关系，基于对标签先验分布的私有估计构建更优的标签随机化器。实验证明，这些随机化器在多个数据集上实现了当前最优的隐私-效用权衡，凸显了在标签差分隐私神经网络训练中降低偏差的重要性。我们还提供了理论结果，揭示了最优无偏随机化器的结构特性。