Parameter-efficient fine-tuning (PEFT) methods have emerged as a practical solution for adapting large foundation models to downstream tasks, reducing computational and memory costs by updating only a small subset of parameters. Among them, approaches like LoRA aim to strike a balance between efficiency and expressiveness, but often suffer from slow convergence and limited adaptation capacity due to their inherent low-rank constraints. This trade-off hampers the ability of PEFT methods to capture complex patterns needed for diverse tasks. To address these challenges, we propose FRoD, a novel fine-tuning method that combines hierarchical joint decomposition with rotational degrees of freedom. By extracting a globally shared basis across layers and injecting sparse, learnable perturbations into scaling factors for flexible full-rank updates, FRoD enhances expressiveness and efficiency, leading to faster and more robust convergence. On 20 benchmarks spanning vision, reasoning, and language understanding, FRoD matches full model fine-tuning in accuracy, while using only 1.72% of trainable parameters under identical training budgets.

翻译：参数高效微调方法已成为将大型基础模型适配到下游任务的实用解决方案，其通过仅更新一小部分参数来降低计算和内存开销。其中，诸如LoRA等方法旨在效率与表达能力之间取得平衡，但由于其固有的低秩约束，往往存在收敛速度慢和适应能力有限的问题。这种权衡阻碍了PEFT方法捕捉多样化任务所需复杂模式的能力。为应对这些挑战，我们提出了FRoD——一种结合分层联合分解与旋转自由度的新型微调方法。通过提取跨层全局共享的基向量，并向缩放因子中注入稀疏、可学习的扰动以实现灵活的全秩更新，FRoD在提升表达能力与效率的同时，实现了更快且更稳健的收敛。在涵盖视觉、推理与语言理解的20个基准测试中，FRoD在相同训练预算下仅使用1.72%的可训练参数，即达到与全模型微调相当的精度。