Federated learning (FL) is a privacy-preserving collaborative learning framework, and differential privacy can be applied to further enhance its privacy protection. Existing FL systems typically adopt Federated Average (FedAvg) as the training algorithm and implement differential privacy with a Gaussian mechanism. However, the inherent privacy-utility trade-off in these systems severely degrades the training performance if a tight privacy budget is enforced. Besides, the Gaussian mechanism requires model weights to be of high-precision. To improve communication efficiency and achieve a better privacy-utility trade-off, we propose a communication-efficient FL training algorithm with differential privacy guarantee. Specifically, we propose to adopt binary neural networks (BNNs) and introduce discrete noise in the FL setting. Binary model parameters are uploaded for higher communication efficiency and discrete noise is added to achieve the client-level differential privacy protection. The achieved performance guarantee is rigorously proved, and it is shown to depend on the level of discrete noise. Experimental results based on MNIST and Fashion-MNIST datasets will demonstrate that the proposed training algorithm achieves client-level privacy protection with performance gain while enjoying the benefits of low communication overhead from binary model updates.

翻译：摘要：联邦学习（FL）是一种隐私保护协同学习框架，可结合差分隐私进一步增强其隐私保护能力。现有联邦学习系统通常采用联邦平均（FedAvg）作为训练算法，并通过高斯机制实现差分隐私。然而，若实施严格的隐私预算，此类系统固有的隐私-效用权衡会严重削弱训练性能。此外，高斯机制要求模型权重具有高精度。为提升通信效率并实现更优的隐私-效用权衡，我们提出一种具备差分隐私保障的通信高效型联邦学习训练算法。具体而言，我们提出在联邦学习场景中采用二元神经网络（BNN）并引入离散噪声。上传二元模型参数可实现更高通信效率，添加离散噪声则可实现客户端级差分隐私保护。该算法性能保障得到严格证明，且表明其性能取决于离散噪声的强度。基于MNIST和Fashion-MNIST数据集的实验结果表明，所提训练算法在实现客户端级隐私保护的同时，能通过二元模型更新带来的低通信开销优势获得性能提升。