Split Federated Learning (SFL) is a distributed machine learning framework which strategically divides the learning process between a server and clients and collaboratively trains a shared model by aggregating local models updated based on data from distributed clients. However, data heterogeneity and partial client participation result in label distribution skew, which severely degrades the learning performance. To address this issue, we propose SFL with Concatenated Activations and Logit Adjustments (SCALA). Specifically, the activations from the client-side models are concatenated as the input of the server-side model so as to centrally adjust label distribution across different clients, and logit adjustments of loss functions on both server-side and client-side models are performed to deal with the label distribution variation across different subsets of participating clients. Theoretical analysis and experimental results verify the superiority of the proposed SCALA on public datasets.
翻译:分裂联邦学习(SFL)是一种分布式机器学习框架,通过将学习过程策略性地分割至服务器与客户端之间,并基于分布式客户端数据聚合本地更新后的模型来协同训练共享模型。然而,数据异构性与部分客户端参与会导致标签分布偏移,严重削弱学习性能。为解决该问题,我们提出基于拼接激活与对数调整的SFL方法(SCALA)。具体而言,将客户端模型的激活值拼接作为服务器端模型的输入,以集中调整不同客户端的标签分布;同时对服务器端与客户端模型的损失函数进行对数调整,应对不同参与客户端子集间的标签分布变化。理论分析与实验结果验证了所提出的SCALA在公开数据集上的优越性。