Low-Rank Adaptation (LoRA) has become a popular solution for fine-tuning large language models (LLMs) in federated settings, dramatically reducing update costs by introducing trainable low-rank matrices. However, when integrated with frameworks like FedIT, LoRA introduces a critical vulnerability: clients submit $A$ and $B$ matrices separately, while only their product $AB$ determines the model update, yet this composite is never directly verified. We propose Gradient Assembly Poisoning (GAP), a novel attack that exploits this blind spot by crafting individually benign $A$ and $B$ matrices whose product yields malicious updates. GAP operates without access to training data or inter-client coordination and remains undetected by standard anomaly detectors. We identify four systemic vulnerabilities in LoRA-based federated systems and validate GAP across LLaMA, ChatGLM, and GPT-2. GAP consistently induces degraded or biased outputs while preserving surface fluency, reducing BLEU by up to 14.5\%, increasing factual and grammatical errors by over 800\%, and maintaining 92.6\% long-form response length. These results reveal a new class of stealthy, persistent threats in distributed LoRA fine-tuning.

翻译：低秩适应（LoRA）已成为联邦学习环境下微调大语言模型（LLM）的流行方案，它通过引入可训练的低秩矩阵显著降低了更新成本。然而，当与FedIT等框架结合时，LoRA引入了一个关键漏洞：客户端分别提交$A$和$B$矩阵，而只有它们的乘积$AB$决定了模型更新，但这一复合结果从未被直接验证。我们提出梯度组装投毒（GAP）攻击，这是一种利用此盲点的新型攻击方法，通过构造各自良性的$A$和$B$矩阵，使其乘积产生恶意更新。GAP攻击无需访问训练数据或客户端间协调，并能规避标准异常检测器的检测。我们识别了基于LoRA的联邦系统中的四个系统性漏洞，并在LLaMA、ChatGLM和GPT-2模型上验证了GAP攻击的有效性。GAP攻击持续导致模型输出性能下降或产生偏差，同时保持表面流畅性，使BLEU分数降低高达14.5\%，事实性和语法错误增加超过800\%，并维持92.6\%的长文本回复长度。这些结果揭示了分布式LoRA微调中一类新的隐蔽且持久的威胁。