Adapting billion-parameter language models to a downstream task is still costly, even with parameter-efficient fine-tuning (PEFT). We re-cast task adaptation as output-distribution alignment: the objective is to steer the output distribution toward the task distribution directly during decoding rather than indirectly through weight updates. Building on this view, we introduce Steering Vector Decoding (SVDecode), a lightweight, PEFT-compatible, and theoretically grounded method. We start with a short warm-start fine-tune and extract a task-aware steering vector from the Kullback-Leibler (KL) divergence gradient between the output distribution of the warm-started and pre-trained models. This steering vector is then used to guide the decoding process to steer the model's output distribution towards the task distribution. We theoretically prove that SVDecode is first-order equivalent to the gradient step of full fine-tuning and derive a globally optimal solution for the strength of the steering vector. Across three tasks and nine benchmarks, SVDecode paired with four standard PEFT methods improves multiple-choice accuracy by up to 5 percentage points and open-ended truthfulness by 2 percentage points, with similar gains (1-2 percentage points) on commonsense datasets without adding trainable parameters beyond the PEFT adapter. SVDecode thus offers a lightweight, theoretically grounded path to stronger task adaptation for large language models. Code is available at https://github.com/dl-m9/SVDecode.

翻译：即使采用参数高效微调（PEFT），使拥有数十亿参数的语言模型适应下游任务仍然成本高昂。我们将任务适应重新定义为输出分布对齐：其目标是在解码过程中直接引导输出分布朝向任务分布，而非通过权重更新间接实现。基于这一视角，我们提出了导向向量解码（SVDecode），这是一种轻量级、兼容PEFT且具有理论依据的方法。我们首先进行短时预热微调，并从预热微调模型与预训练模型的输出分布之间的Kullback-Leibler（KL）散度梯度中提取一个任务感知的导向向量。随后，该导向向量被用于引导解码过程，将模型的输出分布导向任务分布。我们从理论上证明了SVDecode在效果上等价于全参数微调的梯度更新一步，并推导出了导向向量强度的全局最优解。在三个任务和九个基准测试中，SVDecode结合四种标准PEFT方法，将多项选择题准确率提升了高达5个百分点，将开放式问答的真实性提升了2个百分点；在常识数据集上，在不增加PEFT适配器之外可训练参数的情况下，也取得了类似的提升（1-2个百分点）。因此，SVDecode为大语言模型实现更强的任务适应提供了一条轻量级且理论坚实的路径。代码发布于 https://github.com/dl-m9/SVDecode。