Federated Learning (FL) has emerged to allow multiple clients to collaboratively train machine learning models on their private data. However, training and deploying large models for broader applications is challenging in resource-constrained environments. Fortunately, Split Federated Learning (SFL) offers an excellent solution by alleviating the computation and communication burden on the clients SFL often assumes labeled data for local training on clients, however, it is not the case in practice.Prior works have adopted semi-supervised techniques for leveraging unlabeled data in FL, but data non-IIDness poses another challenge to ensure training efficiency. Herein, we propose Pseudo-Clustering Semi-SFL, a novel system for training models in scenarios where labeled data reside on the server. By introducing Clustering Regularization, model performance under data non-IIDness can be improved. Besides, our theoretical and experimental investigations into model convergence reveal that the inconsistent training processes on labeled and unlabeled data impact the effectiveness of clustering regularization. Upon this, we develop a control algorithm for global updating frequency adaptation, which dynamically adjusts the number of supervised training iterations to mitigate the training inconsistency. Extensive experiments on benchmark models and datasets show that our system provides a 3.3x speed-up in training time and reduces the communication cost by about 80.1% while reaching the target accuracy, and achieves up to 6.9% improvement in accuracy under non-IID scenarios compared to the state-of-the-art.

翻译：联邦学习（FL）应运而生，允许多个客户端在私有数据上协作训练机器学习模型。然而，在资源受限的环境中训练和部署大规模模型以服务于更广泛的应用仍面临挑战。幸运的是，分裂联邦学习（SFL）通过减轻客户端的计算和通信负担提供了出色的解决方案。SFL通常假设客户端拥有用于本地训练的标注数据，但这在实际中并不成立。先前的工作已在FL中采用半监督技术来利用未标注数据，但数据非独立同分布（non-IIDness）对训练效率构成了另一挑战。为此，我们提出Pseudo-Clustering Semi-SFL，一种新颖的系统，用于在标注数据驻留在服务器端的场景下训练模型。通过引入聚类正则化，可以提升数据非独立同分布下的模型性能。此外，我们对模型收敛性的理论和实验研究表明，标注数据与未标注数据上不一致的训练过程会影响聚类正则化的有效性。基于此，我们开发了一种用于全局更新频率自适应的控制算法，该算法动态调整监督训练迭代次数以缓解训练不一致性。在基准模型和数据集上的大量实验表明，我们的系统在达到目标精度的同时，训练速度提升了3.3倍，通信成本降低了约80.1%，并且在非独立同分布场景下准确率较现有最优方法提升了高达6.9%。