Estimating the uncertainty of responses of Large Language Models~(LLMs) remains a critical challenge. While recent Bayesian methods have demonstrated effectiveness in quantifying uncertainty through low-rank weight updates, they typically require complex fine-tuning or post-training procedures. In this paper, we propose Training-Free Bayesianization~(TFB), a novel framework that transforms existing off-the-shelf trained LoRA adapters into Bayesian ones without additional training. TFB systematically searches for the maximally acceptable level of variance in the weight posterior, constrained within a family of low-rank isotropic Gaussian distributions. We theoretically demonstrate that under mild conditions, this search process is equivalent to variational inference for the weights. Through comprehensive experiments, we show that TFB achieves superior uncertainty estimation and generalization compared to existing methods while eliminating the need for complex training procedures. Code will be available at https://github.com/Wang-ML-Lab/bayesian-peft.

翻译：估计大语言模型（LLMs）响应的不确定性仍然是一个关键挑战。虽然近期的贝叶斯方法已通过低秩权重更新在量化不确定性方面展现出有效性，但这些方法通常需要复杂的微调或后训练流程。本文提出无需训练的贝叶斯化（TFB）——一种创新框架，可将现有预训练的低秩适配器（LoRA）直接转换为贝叶斯版本，无需额外训练。TFB系统性地在低秩各向同性高斯分布族约束下，搜索权重后验分布中可接受的最大方差水平。我们从理论上证明，在温和条件下，该搜索过程等价于权重的变分推断。通过全面实验，我们表明TFB在消除复杂训练流程需求的同时，相较于现有方法实现了更优的不确定性估计与泛化性能。代码发布于https://github.com/Wang-ML-Lab/bayesian-peft。