Can we find a network architecture for ML model training so as to optimize training loss (and thus, accuracy) in Split Federated Learning (SFL)? And can this architecture also reduce training delay and communication overhead? While accuracy is not influenced by how we split the model in ordinary, state-of-the-art SFL, in this work we answer the questions above in the affirmative. Recent Hierarchical SFL (HSFL) architectures adopt a three-tier training structure consisting of clients, (local) aggregators, and a central server. In this architecture, the model is partitioned at two partitioning layers into three sub-models, which are executed across the three tiers. Despite their merits, HSFL architectures overlook the impact of the partitioning layers and client-to-aggregator assignments on accuracy, delay, and overhead. This work explicitly captures the impact of the partitioning layers and client-to-aggregator assignments on accuracy, delay and overhead by formulating a joint optimization problem. We prove that the problem is NP-hard and propose the first accuracy-aware heuristic algorithm that explicitly accounts for model accuracy, while remaining delay-efficient. Simulation results on public datasets show that our approach can improve accuracy by 3%, while reducing delay by 20% and overhead by 50%, compared to state-of-the-art SFL and HSFL schemes.

翻译：我们能否找到一种用于机器学习模型训练的网络架构，以优化分割联邦学习中的训练损失（从而提升精度）？并且这种架构能否同时降低训练延迟和通信开销？尽管在现有先进的分割联邦学习中，模型分割方式通常不影响精度，但本工作对上述问题给出了肯定回答。近期提出的层次化分割联邦学习架构采用包含客户端、（本地）聚合器与中央服务器的三层训练结构。该架构通过在两个分割层将模型划分为三个子模型，并分别部署于三个层级执行。尽管具有优势，现有层次化分割联邦学习架构忽略了分割层选择与客户端-聚合器分配对精度、延迟及开销的影响。本研究通过构建联合优化问题，明确揭示了分割层选择与客户端-聚合器分配对精度、延迟及开销的作用机制。我们证明了该问题是NP难问题，并提出了首个显式考虑模型精度且保持延迟高效性的精度感知启发式算法。在公开数据集上的仿真结果表明，相较于现有先进的分割联邦学习与层次化分割联邦学习方案，我们的方法可将精度提升3%，同时降低20%的延迟并减少50%的开销。