Parameter-Efficient Fine-Tuning (PEFT) has become essential for adapting foundation models to downstream NLP tasks. However, current PEFT methods often struggle with robustness to noise and performance degradation on limited training data. We propose SDBN (Small Data Big Noise), a unified framework that brings adversarial training to PEFT - a combination that remains less studied in the PEFT setting despite its complementary strengths - to enhance model robustness and generalization, outperforming alternative approaches. We also introduce two variants of the method that use discrete uncertainty sets: SDBN-h, which enumerates character-level edits and selects worst-case variants using gradients, and SDBN-p, which uses LLM-generated variants for robust optimization in generative tasks. Experiments across multiple benchmarks reveal substantial improvements, particularly in low-resource settings and under both word-level and character-level corruptions. This framework addresses the less explored intersection of adversarial training and parameter-efficient adaptation, without introducing additional parameters or only modest computational overhead, making PEFT deployments more reliable in real-world scenarios where data scarcity and linguistic variability often coexist

翻译：参数高效微调（PEFT）已成为将基础模型适配到下游自然语言处理任务的关键技术。然而，当前的PEFT方法在处理噪声鲁棒性以及有限训练数据下的性能退化方面仍面临挑战。我们提出SDBN（小数据大噪声）统一框架，将对抗训练引入PEFT——尽管两者具有互补优势，但这一结合在PEFT场景中鲜有研究——从而增强模型的鲁棒性与泛化能力，其性能优于现有替代方法。我们还引入了该方法基于离散不确定集的两个变体：SDBN-h通过枚举字符级编辑并利用梯度选择最差变体，而SDBN-p则利用大语言模型生成的变体进行生成式任务的鲁棒优化。在多个基准上的实验表明，该方法在低资源设置以及词级和字符级扰动条件下均取得了显著改进。该框架探索了对抗训练与参数高效适配中较少被研究的交叉领域，且无需引入额外参数或仅需少量计算开销，从而使得PEFT部署在数据稀缺与语言变异并存的现实场景中更加可靠。