Federated Learning (FL) has shown considerable promise in Machine Learning (ML) across numerous devices for privacy protection, efficient data utilization, and dynamic collaboration. However, mobile devices typically have limited and heterogeneous computational capabilities, and different devices may even have different tasks. This client heterogeneity is a major bottleneck hindering the practical application of FL. Existing work mainly focuses on mitigating FL's computation and communication overhead of a single task while overlooking the computing resource heterogeneity issue of different devices in FL. To tackle this, we design FedAPTA, a federated multi-task learning framework. FedAPTA overcomes computing resource heterogeneity through the developed layer-wise model pruning technique, which reduces local model size while considering both data and device heterogeneity. To aggregate structurally heterogeneous local models of different tasks, we introduce a heterogeneous model recovery strategy and a task-aware model aggregation method that enables the aggregation through infilling local model architecture with the shared global model and clustering local models according to their specific tasks. We deploy FedAPTA on a realistic FL platform and benchmark it against nine SOTA FL methods. The experimental outcomes demonstrate that the proposed FedAPTA considerably outperforms the state-of-the-art FL methods by up to 4.23\%. Our code is available at https://github.com/Zhenzovo/FedAPTA.

翻译：联邦学习（FL）在跨设备机器学习（ML）中展现出巨大潜力，能够实现隐私保护、高效数据利用和动态协作。然而，移动设备通常具有有限且异构的计算能力，不同设备甚至可能执行不同的任务。这种客户端异构性是阻碍联邦学习实际应用的主要瓶颈。现有研究主要集中于降低联邦学习中单一任务的计算与通信开销，却忽视了不同设备在计算资源异构性方面的问题。为解决这一挑战，我们设计了FedAPTA——一种联邦多任务学习框架。FedAPTA通过开发的分层模型剪枝技术克服计算资源异构性，该技术在考虑数据和设备异质性的同时减小本地模型规模。为聚合不同任务的结构异构本地模型，我们引入了异构模型恢复策略和任务感知模型聚合方法：前者通过共享全局模型填充本地模型架构实现聚合，后者则依据具体任务对本地模型进行聚类。我们在真实的联邦学习平台上部署FedAPTA，并与九种最先进的联邦学习方法进行基准测试。实验结果表明，所提出的FedAPTA显著优于现有最优方法，性能提升最高达4.23%。代码已开源：https://github.com/Zhenzovo/FedAPTA。