Adapting billion-parameter language models to a downstream task is still costly, even with parameter-efficient fine-tuning (PEFT). We re-cast task adaptation as output-distribution alignment: the objective is to steer the output distribution toward the task distribution directly during decoding rather than indirectly through weight updates. Building on this view, we introduce Steering Vector Decoding (SVDecode), a lightweight, PEFT-compatible, and theoretically grounded method. We start with a short warm-start fine-tune and extract a task-aware steering vector from the Kullback-Leibler (KL) divergence gradient between the output distribution of the warm-started and pre-trained models. This steering vector is then used to guide the decoding process to steer the model's output distribution towards the task distribution. We theoretically prove that SVDecode is first-order equivalent to the gradient step of full fine-tuning and derive a globally optimal solution for the strength of the steering vector. Across three tasks and nine benchmarks, SVDecode paired with four standard PEFT methods improves multiple-choice accuracy by up to 5 percentage points and open-ended truthfulness by 2 percentage points, with similar gains (1-2 percentage points) on commonsense datasets without adding trainable parameters beyond the PEFT adapter. SVDecode thus offers a lightweight, theoretically grounded path to stronger task adaptation for large language models. Code is available at https://github.com/dl-m9/SVDecode.

翻译：适配数十亿参数的语言模型至下游任务仍然成本高昂，即使采用参数高效微调（PEFT）方法。本文将任务适配重新定义为输出分布对齐：其目标是在解码过程中直接将输出分布导向任务分布，而非通过权重更新间接实现。基于这一视角，我们提出了导向向量解码（SVDecode），一种轻量级、兼容PEFT且具有理论依据的方法。我们首先进行短时预热微调，并从预热微调模型与预训练模型的输出分布之间的Kullback-Leibler（KL）散度梯度中提取一个任务感知的导向向量。随后，该导向向量被用于引导解码过程，将模型的输出分布导向任务分布。我们从理论上证明了SVDecode在效果上等同于全量微调的一阶梯度步，并推导出了导向向量强度的全局最优解。在三个任务和九个基准测试中，SVDecode与四种标准PEFT方法结合，将多项选择准确率提升了高达5个百分点，将开放式生成的真实性提升了2个百分点；在常识数据集上，仅使用PEFT适配器而未增加可训练参数的情况下，也取得了类似的提升（1-2个百分点）。因此，SVDecode为大语言模型提供了通向更强任务适配能力的轻量级、理论驱动的路径。代码可在 https://github.com/dl-m9/SVDecode 获取。