Training large language models (LLMs) at the network edge faces fundamental challenges arising from device resource constraints, severe data heterogeneity, and heightened privacy risks. To address these, we propose ELSA (Efficient LLM-centric Split Aggregation), a novel framework that systematically integrates split learning (SL) and hierarchical federated learning (HFL) for distributed LLM fine-tuning over resource-constrained edge networks. ELSA introduces three key innovations. First, it employs a task-agnostic, behavior-aware client clustering mechanism that constructs semantic fingerprints using public probe inputs and symmetric KL divergence, further enhanced by prediction-consistency-based trust scoring and latency-aware edge assignment to jointly address data heterogeneity, client unreliability, and communication constraints. Second, it splits the LLM into three parts across clients and edge servers, with the cloud used only for adapter aggregation, enabling an effective balance between on-device computation cost and global convergence stability. Third, it incorporates a lightweight communication scheme based on computational sketches combined with semantic subspace orthogonal perturbation (SS-OP) to reduce communication overhead while mitigating privacy leakage during model exchanges. Experiments across diverse NLP tasks demonstrate that ELSA consistently outperforms state-of-the-art methods in terms of adaptability, convergence behavior, and robustness, establishing a scalable and privacy-aware solution for edge-side LLM fine-tuning under resource constraints.

翻译：在网络边缘训练大型语言模型（LLM）面临设备资源受限、数据异构性严重及隐私风险加剧等根本性挑战。为应对这些问题，本文提出ELSA（高效LLM中心化分割聚合）——一种创新框架，通过系统整合分割学习（SL）与分层联邦学习（HFL），实现在资源受限边缘网络上的分布式LLM微调。ELSA包含三项核心创新：首先，采用任务无关的行为感知客户端聚类机制，通过公共探针输入与对称KL散度构建语义指纹，并辅以基于预测一致性的信任评分与延迟感知边缘节点分配，协同解决数据异构性、客户端不可靠性及通信约束问题。其次，将LLM分割为三个部分分别部署于客户端与边缘服务器，云端仅负责适配器聚合，从而在设备端计算成本与全局收敛稳定性之间实现有效平衡。第三，引入基于计算草图与语义子空间正交扰动（SS-OP）的轻量级通信方案，在降低通信开销的同时缓解模型交换过程中的隐私泄露风险。跨多类NLP任务的实验表明，ELSA在适应性、收敛行为与鲁棒性方面均持续优于现有先进方法，为资源受限环境下边缘侧LLM微调提供了可扩展且隐私感知的解决方案。