Split federated learning (SFL) is a compute-efficient paradigm in distributed machine learning (ML), where components of large ML models are outsourced to remote servers. A significant challenge in SFL, particularly when deployed over wireless channels, is the susceptibility of transmitted model parameters to adversarial jamming that could jeopardize the learning process. This is particularly pronounced for word embedding parameters in large language models (LLMs), which are crucial for language understanding. In this paper, rigorous insights are provided into the influence of jamming LLM word embeddings in SFL by deriving an expression for the ML training loss divergence and showing that it is upper-bounded by the mean squared error (MSE). Based on this analysis, a physical layer framework is developed for resilient SFL with LLMs (R-SFLLM) over wireless networks. R-SFLLM leverages wireless sensing data to gather information on the jamming directions-of-arrival (DoAs) for the purpose of devising a novel, sensing-assisted anti-jamming strategy while jointly optimizing beamforming, user scheduling, and resource allocation. Extensive experiments using BERT and RoBERTa models demonstrate R-SFLLM's effectiveness, achieving close-to-baseline performance across various natural language processing (NLP) tasks and datasets. The proposed methodology further introduces an adversarial training component, where controlled noise exposure significantly enhances the LLM's resilience to perturbed parameters during training. The results show that more noise-sensitive models, such as RoBERTa, benefit from this feature, especially when resource allocation is unfair. It is also shown that worst-case jamming in particular translates into worst-case model outcomes, thereby necessitating the need for jamming-resilient SFL protocols.

翻译：分割联邦学习（SFL）是分布式机器学习（ML）中一种计算高效的范式，它将大型ML模型的组件外包给远程服务器。SFL面临的一个重大挑战，尤其是在无线信道上部署时，是传输的模型参数容易受到对抗性干扰，这可能危及学习过程。这对于大型语言模型（LLM）中的词嵌入参数尤为突出，这些参数对语言理解至关重要。本文通过推导ML训练损失散度的表达式，并证明其上界为均方误差（MSE），从而对SFL中LLM词嵌入受干扰的影响提供了严谨的见解。基于此分析，我们为无线网络上的LLM弹性SFL（R-SFLLM）开发了一个物理层框架。R-SFLLM利用无线传感数据收集干扰到达方向（DoA）信息，旨在设计一种新颖的传感辅助抗干扰策略，同时联合优化波束成形、用户调度和资源分配。使用BERT和RoBERTa模型的大量实验证明了R-SFLLM的有效性，在各种自然语言处理（NLP）任务和数据集上实现了接近基线的性能。所提出的方法进一步引入了对抗训练组件，其中受控的噪声暴露显著增强了LLM在训练期间对参数扰动的弹性。结果表明，对噪声更敏感的模型（如RoBERTa）从此特性中受益，尤其是在资源分配不公的情况下。研究还表明，最坏情况的干扰尤其会导致最坏的模型结果，因此需要抗干扰的SFL协议。