Large pre-trained models achieve remarkable success across diverse domains, yet fully fine-tuning incurs prohibitive computational and memory costs. Parameter-efficient fine-tuning (PEFT) has thus become a mainstream paradigm. Among them, Low-Rank Adaptation (LoRA) introduces trainable low-rank matrices and shows strong performance, nevertheless, its fixed-rank design limits flexibility. Dynamic rank allocation methods mitigate this issue by pruning redundant directions; however, they often rely on heuristic, element-level metrics that globally sort rank directions without matrix-wise distinction, and they lack mechanisms to expand capacity in layers requiring additional adaptation. To overcome these limitations, we propose FlexLoRA, an entropy-guided flexible low-rank adaptation framework that (i) evaluates matrix importance via spectral energy entropy, (ii) supports rank pruning and expansion under a global budget, and (iii) employs zero-impact initialization for newly added singular directions to ensure stability. By addressing granularity, flexibility, and stability limitations, FlexLoRA provides a more principled solution for PEFT. Extensive experiments show that FlexLoRA consistently outperforms state-of-the-art baselines across benchmarks. Codes are available at https://github.com/Chongjie-Si/Subspace-Tuning.

翻译：大规模预训练模型在众多领域取得了显著成功，然而完全微调会带来极高的计算与内存开销。参数高效微调因此成为主流范式。其中，低秩自适应通过引入可训练的低秩矩阵展现出强大性能，但其固定秩的设计限制了灵活性。动态秩分配方法通过剪枝冗余方向缓解了这一问题；然而，这些方法通常依赖启发式的元素级度量指标，对秩方向进行全局排序而缺乏矩阵层面的区分，并且缺少在需要额外适应性的层中扩展容量的机制。为克服这些局限，我们提出FlexLoRA，一种熵引导的灵活低秩自适应框架，其具备以下特点：（i）通过谱能量熵评估矩阵重要性，（ii）在全局预算下支持秩的剪枝与扩展，（iii）对新添加的奇异方向采用零影响初始化以确保稳定性。通过解决粒度、灵活性与稳定性方面的限制，FlexLoRA为参数高效微调提供了一个更具原则性的解决方案。大量实验表明，FlexLoRA在多个基准测试中均持续优于现有先进基线方法。代码公开于https://github.com/Chongjie-Si/Subspace-Tuning。