Low-rank adaptation (LoRA) is a standard approach for fine-tuning large language models, yet its many variants report conflicting empirical gains, often on the same benchmarks. We show that these contradictions arise from a single overlooked factor: the batch size. When properly tuned, vanilla LoRA often matches the performance of more complex variants. We further propose a proxy-based, cost-efficient strategy for batch size tuning, revealing the impact of rank, dataset size, and model capacity on the optimal batch size. Our findings elevate batch size from a minor implementation detail to a first-order design parameter, reconciling prior inconsistencies and enabling more reliable evaluations of LoRA variants.

翻译：低秩适应（LoRA）是微调大语言模型的标准方法，但其众多变体在同一基准测试中常报告相互矛盾的实证增益。我们发现这些矛盾源于一个被忽视的因素：批次大小。当经过适当调优时，原始LoRA通常能与更复杂变体的性能相匹配。我们进一步提出了一种基于代理的、成本高效的批次大小调优策略，揭示了秩、数据集大小和模型容量对最优批次大小的影响。我们的研究将批次大小从一个次要的实现细节提升为一阶设计参数，调和了先前的不一致性，并为LoRA变体的更可靠评估提供了可能。