Low-Rank Adaptation (LoRA) is a fundamental parameter-efficient fine-tuning method that balances efficiency and performance in large-scale neural networks. However, the proliferation of LoRA variants has led to fragmentation in methodology, theory, code, and evaluation. To this end, this work presents the first unified study of LoRA variants, offering a systematic taxonomy, unified theoretical review, structured codebase, and standardized empirical assessment. First, we categorize LoRA variants along four principal axes: rank, optimization dynamics, initialization, and integration with Mixture-of-Experts. Then, we review their relationships and evolution within a common theoretical framework focused on low-rank update dynamics. Further, we introduce LoRAFactory, a modular codebase that implements variants through a unified interface, supporting plug-and-play experimentation and fine-grained analysis. Last, using this codebase, we conduct a large-scale evaluation across natural language generation, natural language understanding, and image classification tasks, systematically exploring key hyperparameters. Our results uncover several findings, notably: LoRA and its variants exhibit pronounced sensitivity to the choices of learning rate compared to other hyperparameters; moreover, with proper hyperparameter configurations, LoRA consistently matches or surpasses the performance of most of its variants.

翻译：低秩适应（LoRA）是一种基础的参数高效微调方法，能在大型神经网络中平衡效率与性能。然而，LoRA变体的激增导致了方法、理论、代码和评估的碎片化。为此，本研究首次对LoRA变体进行了统一研究，提供了系统化的分类法、统一的理论综述、结构化的代码库和标准化的实证评估。首先，我们沿四个主要维度对LoRA变体进行分类：秩、优化动态、初始化以及与专家混合（Mixture-of-Experts）的集成。接着，我们在专注于低秩更新动态的统一理论框架内，综述了这些变体之间的关系与演进。进一步地，我们引入了LoRAFactory——一个通过统一接口实现各变体的模块化代码库，支持即插即用的实验和细粒度分析。最后，利用该代码库，我们在自然语言生成、自然语言理解和图像分类任务上进行了大规模评估，系统性地探索了关键超参数。我们的结果揭示了若干发现，特别是：相较于其他超参数，LoRA及其变体对学习率的选择表现出显著的敏感性；此外，在适当的超参数配置下，LoRA的性能始终匹配或超越其大多数变体。