Heterogeneous federated multi-task learning (HFMTL) is a federated learning technique that combines heterogeneous tasks of different clients to achieve more accurate, comprehensive predictions. In real-world applications, visual and natural language tasks typically require large-scale models to extract high-level abstract features. However, large-scale models cannot be directly applied to existing federated multi-task learning methods. Existing HFML methods also disregard the impact of gradient conflicts on multi-task optimization during the federated aggregation process. In this work, we propose an innovative framework called FedBone, which enables the construction of large-scale models with better generalization from the perspective of server-client split learning and gradient projection. We split the entire model into two components: a large-scale general model (referred to as the general model) on the cloud server and multiple task-specific models (referred to as the client model) on edge clients, solving the problem of insufficient computing power on edge clients. The conflicting gradient projection technique is used to enhance the generalization of the large-scale general model between different tasks. The proposed framework is evaluated on two benchmark datasets and a real ophthalmic dataset. Comprehensive results demonstrate that FedBone efficiently adapts to heterogeneous local tasks of each client and outperforms existing federated learning algorithms in most dense prediction and classification tasks with off-the-shelf computational resources on the client side.

翻译：摘要：异构联邦多任务学习（HFMTL）是一种联邦学习技术，通过整合不同客户端的异构任务，实现更准确、全面的预测。在实际应用中，视觉和自然语言任务通常需要大规模模型来提取高层抽象特征。然而，现有联邦多任务学习方法无法直接应用大规模模型。现有HFML方法在联邦聚合过程中也忽略了梯度冲突对多任务优化的影响。本文提出了一种创新框架FedBone，该框架从服务器-客户端分离学习与梯度投影的角度，实现了泛化能力更强的大规模模型构建。我们将整个模型拆分为两个组件：云服务器上的大规模通用模型（简称通用模型）和边缘客户端上的多个任务特定模型（简称客户端模型），从而解决了边缘客户端计算能力不足的问题。采用冲突梯度投影技术来增强大规模通用模型在不同任务间的泛化能力。该框架在两个基准数据集和一个真实眼科数据集上进行了评估。综合结果表明，在利用客户端现有计算资源的情况下，FedBone能有效适应每个客户端的异构本地任务，并在大多数密集预测与分类任务中优于现有联邦学习算法。