Conventionally, federated learning aims to optimize a single objective, typically the utility. However, for a federated learning system to be trustworthy, it needs to simultaneously satisfy multiple/many objectives, such as maximizing model performance, minimizing privacy leakage and training cost, and being robust to malicious attacks. Multi-Objective Optimization (MOO) aiming to optimize multiple conflicting objectives at the same time is quite suitable for solving the optimization problem of Trustworthy Federated Learning (TFL). In this paper, we unify MOO and TFL by formulating the problem of constrained multi-objective federated learning (CMOFL). Under this formulation, existing MOO algorithms can be adapted to TFL straightforwardly. Different from existing CMOFL works focusing on utility, efficiency, fairness, and robustness, we consider optimizing privacy leakage along with utility loss and training cost, the three primary objectives of a TFL system. We develop two improved CMOFL algorithms based on NSGA-II and PSL, respectively, for effectively and efficiently finding Pareto optimal solutions, and we provide theoretical analysis on their convergence. We design specific measurements of privacy leakage, utility loss, and training cost for three privacy protection mechanisms: Randomization, BatchCrypt (An efficient version of homomorphic encryption), and Sparsification. Empirical experiments conducted under each of the three protection mechanisms demonstrate the effectiveness of our proposed algorithms.

翻译：传统上，联邦学习旨在优化单一目标，通常为模型效用。然而，一个值得信任的联邦学习系统需要同时满足多个/诸多目标，例如最大化模型性能、最小化隐私泄露风险与训练成本，以及对恶意攻击具有鲁棒性。多目标优化（MOO）旨在同时优化多个相互冲突的目标，非常适用于解决可信联邦学习（TFL）中的优化问题。本文通过构建受限多目标联邦学习（CMOFL）问题，将多目标优化与可信联邦学习统一起来。在该问题框架下，现有多目标优化算法可直接适用于可信联邦学习。与现有聚焦于效用、效率、公平性和鲁棒性的CMOFL研究不同，我们同时考虑了隐私泄露、效用损失和训练成本这三个可信联邦学习系统的核心目标。我们基于NSGA-II和PSL分别开发了两种改进型CMOFL算法，以高效且有效地寻找帕累托最优解，并提供了收敛性理论分析。针对随机化、BatchCrypt（同态加密的高效版本）和稀疏化这三种隐私保护机制，我们设计了隐私泄露、效用损失和训练成本的具体度量方法。在每种保护机制下进行的实证实验证明了所提算法的有效性。