Despite the notable success of language models (LMs) in various natural language processing (NLP) tasks, the reliability of LMs is susceptible to backdoor attacks. Prior research attempts to mitigate backdoor learning while training the LMs on the poisoned dataset, yet struggles against complex backdoor attacks in real-world scenarios. In this paper, we investigate the learning mechanisms of backdoor LMs in the frequency space by Fourier analysis. Our findings indicate that the backdoor mapping presented on the poisoned datasets exhibits a more discernible inclination towards lower frequency compared to clean mapping, resulting in the faster convergence of backdoor mapping. To alleviate this dilemma, we propose Multi-Scale Low-Rank Adaptation (MuScleLoRA), which deploys multiple radial scalings in the frequency space with low-rank adaptation to the target model and further aligns the gradients when updating parameters. Through downscaling in the frequency space, MuScleLoRA encourages the model to prioritize the learning of relatively high-frequency clean mapping, consequently mitigating backdoor learning. Experimental results demonstrate that MuScleLoRA outperforms baselines significantly. Notably, MuScleLoRA reduces the average success rate of diverse backdoor attacks to below 15\% across multiple datasets and generalizes to various backbone LMs, including BERT, RoBERTa, and Llama2. The codes are available at https://github.com/ZrW00/MuScleLoRA.

翻译：尽管语言模型（LMs）在多种自然语言处理（NLP）任务中取得了显著成功，但其可靠性容易受到后门攻击的影响。以往的研究试图在投毒数据集上训练语言模型时减轻后门学习，但在现实场景中对抗复杂后门攻击时仍面临挑战。本文通过傅里叶分析从频率空间探讨后门语言模型的学习机制。我们的发现表明，投毒数据集上呈现的后门映射相比干净映射表现出更明显的低频偏好，导致后门映射收敛速度更快。为解决这一困境，我们提出多尺度低秩适应（MuScleLoRA），该方法在频率空间部署多个径向缩放，通过低秩适应目标模型，并在参数更新时进一步对齐梯度。通过在频率空间进行降低缩放，MuScleLoRA促使模型优先学习相对高频的干净映射，从而减轻后门学习。实验结果表明，MuScleLoRA显著优于基线方法。值得注意的是，MuScleLoRA将多种后门攻击在多个数据集上的平均成功率降至15%以下，并适用于多种骨干语言模型，包括BERT、RoBERTa和Llama2。代码已在https://github.com/ZrW00/MuScleLoRA开源。