Conventionally, federated learning aims to optimize a single objective, typically the utility. However, for a federated learning system to be trustworthy, it needs to simultaneously satisfy multiple/many objectives, such as maximizing model performance, minimizing privacy leakage and training cost, and being robust to malicious attacks. Multi-Objective Optimization (MOO) aiming to optimize multiple conflicting objectives at the same time is quite suitable for solving the optimization problem of Trustworthy Federated Learning (TFL). In this paper, we unify MOO and TFL by formulating the problem of constrained multi-objective federated learning (CMOFL). Under this formulation, existing MOO algorithms can be adapted to TFL straightforwardly. Different from existing CMOFL works focusing on utility, efficiency, fairness, and robustness, we consider optimizing privacy leakage along with utility loss and training cost, the three primary objectives of a TFL system. We develop two improved CMOFL algorithms based on NSGA-II and PSL, respectively, for effectively and efficiently finding Pareto optimal solutions, and we provide theoretical analysis on their convergence. We design specific measurements of privacy leakage, utility loss, and training cost for three privacy protection mechanisms: Randomization, BatchCrypt (An efficient version of homomorphic encryption), and Sparsification. Empirical experiments conducted under each of the three protection mechanisms demonstrate the effectiveness of our proposed algorithms.

翻译：传统上，联邦学习旨在优化单一目标，即效用。然而，可信联邦学习系统需要同时满足多个/众多目标，例如最大化模型性能、最小化隐私泄露和训练成本，并具备对恶意攻击的鲁棒性。多目标优化（MOO）旨在同时优化多个相互冲突的目标，非常适合解决可信联邦学习（TFL）的优化问题。本文通过提出约束多目标联邦学习（CMOFL）问题，将MOO与TFL统一起来。在此框架下，现有MOO算法可直接适配到TFL场景。与现有聚焦于效用、效率、公平性和鲁棒性的CMOFL研究不同，我们同时考虑优化隐私泄露、效用损失和训练成本——这三个TFL系统的核心目标。我们基于NSGA-II和PSL分别开发了两种改进的CMOFL算法，以高效寻找帕累托最优解，并对其收敛性进行了理论分析。针对三种隐私保护机制：随机化、BatchCrypt（同态加密的高效变体）和稀疏化，我们设计了隐私泄露、效用损失和训练成本的具体度量方法。在每种保护机制下进行的实证实验证明了所提算法的有效性。