Federated learning (FL) is a popular distributed machine learning (ML) paradigm, but is often limited by significant communication costs and edge device computation capabilities. Federated Split Learning (FSL) preserves the parallel model training principle of FL, with a reduced device computation requirement thanks to splitting the ML model between the server and clients. However, FSL still incurs very high communication overhead due to transmitting the smashed data and gradients between the clients and the server in each global round. Furthermore, the server has to maintain separate models for every client, resulting in a significant computation and storage requirement that grows linearly with the number of clients. This paper tries to solve these two issues by proposing a communication and storage efficient federated and split learning (CSE-FSL) strategy, which utilizes an auxiliary network to locally update the client models while keeping only a single model at the server, hence avoiding the communication of gradients from the server and greatly reducing the server resource requirement. Communication cost is further reduced by only sending the smashed data in selected epochs from the clients. We provide a rigorous theoretical analysis of CSE-FSL that guarantees its convergence for non-convex loss functions. Extensive experimental results demonstrate that CSE-FSL has a significant communication reduction over existing FSL techniques while achieving state-of-the-art convergence and model accuracy, using several real-world FL tasks.

翻译：联邦学习（FL）是一种流行的分布式机器学习范式，但常受限于显著的通信成本和边缘设备的计算能力。联邦拆分学习（FSL）保留了FL的并行模型训练原理，通过将机器学习模型在服务器与客户端之间拆分，降低了对设备计算能力的需求。然而，FSL每轮全局训练中需在客户端与服务器间传输粉碎数据和梯度，导致极高的通信开销。此外，服务器需为每个客户端维护独立模型，导致计算与存储需求随客户端数量线性增长。本文提出一种通信与存储高效的联邦拆分学习（CSE-FSL）策略，通过利用辅助网络在本地更新客户端模型，同时服务器仅保留单一模型，从而避免服务器端梯度通信并大幅降低服务器资源需求。通过仅在选定轮次中由客户端传输粉碎数据，进一步降低通信成本。我们为CSE-FSL提供了严格的收敛性理论分析，证明其非凸损失函数下的收敛性。大量实验结果表明，在多个真实联邦学习任务中，CSE-FSL相比现有FSL技术显著降低通信成本，同时实现最先进的收敛速度和模型精度。