Federated learning (FL) is a new distributed learning paradigm, with privacy, utility, and efficiency as its primary pillars. Existing research indicates that it is unlikely to simultaneously attain infinitesimal privacy leakage, utility loss, and efficiency. Therefore, how to find an optimal trade-off solution is the key consideration when designing the FL algorithm. One common way is to cast the trade-off problem as a multi-objective optimization problem, i.e., the goal is to minimize the utility loss and efficiency reduction while constraining the privacy leakage not exceeding a predefined value. However, existing multi-objective optimization frameworks are very time-consuming, and do not guarantee the existence of the Pareto frontier, this motivates us to seek a solution to transform the multi-objective problem into a single-objective problem because it is more efficient and easier to be solved. To this end, in this paper, we propose FedPAC, a unified framework that leverages PAC learning to quantify multiple objectives in terms of sample complexity, such quantification allows us to constrain the solution space of multiple objectives to a shared dimension, so that it can be solved with the help of a single-objective optimization algorithm. Specifically, we provide the results and detailed analyses of how to quantify the utility loss, privacy leakage, privacy-utility-efficiency trade-off, as well as the cost of the attacker from the PAC learning perspective.

翻译：联邦学习（FL）是一种新兴的分布式学习范式，隐私性、效用性和效率是其三大核心支柱。现有研究表明，同时实现无穷小的隐私泄露、效用损失和效率损失几乎不可能。因此，如何在设计FL算法时找到最优权衡方案成为关键考量。常见方法是将权衡问题转化为多目标优化问题，即目标是最小化效用损失和效率降低，同时约束隐私泄露不超过预设值。然而，现有的大多数多目标优化框架非常耗时，且无法保证Pareto前沿的存在性。这促使我们寻求一种将多目标问题转化为单目标问题的解决方案，因为单目标问题更高效且更易求解。为此，本文提出FedPAC这一统一框架，该框架利用PAC学习理论从样本复杂度角度量化多个目标，这种量化方法能将多目标的解空间约束到同一维度，从而借助单目标优化算法进行求解。具体而言，我们从PAC学习视角提供了关于效用损失、隐私泄露、隐私-效用-效率权衡以及攻击者成本的量化结果与详细分析。