Fine-tuning large language models (LLMs) has achieved remarkable success across various NLP tasks, but the substantial memory overhead during backpropagation remains a critical bottleneck, especially as model scales grow. Zeroth-order (ZO) optimization alleviates this issue by estimating gradients through forward passes and Gaussian sampling, avoiding the need for backpropagation. However, conventional ZO methods suffer from high variance in gradient estimation due to their reliance on random perturbations, leading to slow convergence and suboptimal performance. We propose a simple plug-and-play method that incorporates prior-informed perturbations to refine gradient estimation. Our method dynamically computes a guiding vector from Gaussian samples, which directs perturbations toward more informative directions, significantly accelerating convergence compared to standard ZO approaches. We further investigate a greedy perturbation strategy to explore the impact of prior knowledge on gradient estimation. Theoretically, we prove that our gradient estimator achieves stronger alignment with the true gradient direction, enhancing optimization efficiency. Extensive experiments across LLMs of varying scales and architectures demonstrate that our proposed method could seamlessly integrate into existing optimization methods, delivering faster convergence and superior performance. Notably, on the OPT-13B model, our method outperforms traditional ZO optimization across all 11 benchmark tasks and surpasses gradient-based baselines on 9 out of 11 tasks, establishing a robust balance between efficiency and accuracy.

翻译：大语言模型（LLM）的微调在各种自然语言处理任务中取得了显著成功，但反向传播过程中的巨大内存开销仍然是关键瓶颈，尤其随着模型规模的增长。零阶（ZO）优化通过前向传播和高斯采样估计梯度，避免了反向传播的需求，从而缓解了这一问题。然而，传统的ZO方法由于依赖随机扰动，导致梯度估计方差较高，收敛速度慢且性能欠佳。我们提出了一种简单的即插即用方法，通过引入基于先验信息的扰动来改进梯度估计。该方法动态地从高斯样本中计算引导向量，将扰动导向信息更丰富的方向，相比标准ZO方法显著加速了收敛。我们进一步研究了贪婪扰动策略，以探索先验知识对梯度估计的影响。理论上，我们证明了所提出的梯度估计器能够与真实梯度方向实现更强的对齐，从而提升优化效率。在不同规模和架构的LLM上进行的大量实验表明，所提出的方法能够无缝集成到现有优化方法中，实现更快的收敛和更优的性能。值得注意的是，在OPT-13B模型上，我们的方法在所有11个基准任务中均优于传统ZO优化，并在11个任务中的9个上超越了基于梯度的基线方法，在效率与精度之间建立了稳健的平衡。