Federated learning (FL) is a common and practical framework for learning a machine model in a decentralized fashion. A primary motivation behind this decentralized approach is data privacy, ensuring that the learner never sees the data of each local source itself. Federated learning then comes with two majors challenges: one is handling potentially complex model updates between a server and a large number of data sources; the other is that de-centralization may, in fact, be insufficient for privacy, as the local updates themselves can reveal information about the sources' data. To address these issues, we consider an approach to federated learning that combines quantization and differential privacy. Absent privacy, Federated Learning often relies on quantization to reduce communication complexity. We build upon this approach and develop a new algorithm called the \textbf{R}andomized \textbf{Q}uantization \textbf{M}echanism (RQM), which obtains privacy through a two-levels of randomization. More precisely, we randomly sub-sample feasible quantization levels, then employ a randomized rounding procedure using these sub-sampled discrete levels. We are able to establish that our results preserve ``Renyi differential privacy'' (Renyi DP). We empirically study the performance of our algorithm and demonstrate that compared to previous work it yields improved privacy-accuracy trade-offs for DP federated learning. To the best of our knowledge, this is the first study that solely relies on randomized quantization without incorporating explicit discrete noise to achieve Renyi DP guarantees in Federated Learning systems.

翻译：联邦学习是一种常见的实用框架，用于以去中心化方式学习机器学习模型。这种去中心化方法的主要动机之一是数据隐私，确保学习者永远不会看到每个本地数据源本身的数据。联邦学习随之面临两大挑战：一是处理服务器与大量数据源之间潜在复杂的模型更新；二是去中心化可能实际上不足以保护隐私，因为本地更新本身可能泄露数据源的信息。为解决这些问题，我们提出了一种结合量化与差分隐私的联邦学习方法。在无隐私约束的情况下，联邦学习通常依赖量化来降低通信复杂度。我们在此方法基础上开发了一种新算法，称为随机量化机制（RQM），该算法通过两层随机化实现隐私保护。具体而言，我们随机子采样可行的量化级别，然后使用这些子采样的离散级别采用随机舍入过程。我们能够证明，我们的结果保持了“Renyi差分隐私”（Renyi DP）。我们通过实验研究了算法的性能，并表明与先前工作相比，它在差分隐私联邦学习中实现了更优的隐私-精度权衡。据我们所知，这是首次仅依赖随机量化而不引入显式离散噪声，在联邦学习系统中实现Renyi DP保证的研究。