In private federated learning (FL), a server aggregates differentially private updates from a large number of clients in order to train a machine learning model. The main challenge in this setting is balancing privacy with both classification accuracy of the learnt model as well as the number of bits communicated between the clients and server. Prior work has achieved a good trade-off by designing a privacy-aware compression mechanism, called the minimum variance unbiased (MVU) mechanism, that numerically solves an optimization problem to determine the parameters of the mechanism. This paper builds upon it by introducing a new interpolation procedure in the numerical design process that allows for a far more efficient privacy analysis. The result is the new Interpolated MVU mechanism that is more scalable, has a better privacy-utility trade-off, and provides SOTA results on communication-efficient private FL on a variety of datasets.
翻译:在私有联邦学习(FL)中,服务器聚合来自大量客户端的差分隐私更新,以训练机器学习模型。该场景的主要挑战在于平衡隐私性、学习模型的分类精度以及客户端与服务器间通信的比特数。先前的研究通过设计一种名为最小方差无偏(MVU)机制的隐私感知压缩机制,实现了良好的折中——该机制通过数值求解优化问题来确定其参数。本文在此基础上引入了一种新的插值过程,在数值设计过程中实现了更高效的隐私分析。由此产生的新型插值MVU机制更具可扩展性,具有更优的隐私-效用权衡,并在多种数据集上的通信高效私有联邦学习任务中取得了最先进(SOTA)的结果。