Quantifying uncertainty in Large Language Models (LLMs) is essential for mitigating hallucinations and enabling risk-aware deployment in safety-critical tasks. However, estimating Epistemic Uncertainty(EU) via Deep Ensembles is computationally prohibitive at the scale of modern models. We propose a framework that leverages the small draft models to efficiently estimate token-level EU, bypassing the need for full-scale ensembling. Theoretically grounded in a Bias-Variance Decomposition, our approach approximates EU via Jensen-Shannon divergence among drafts (variance proxy) and KL divergence between the draft mixture and the target (bias proxy). To further ensure accuracy without significant overhead, we introduce Online Stochastic Distillation (OSD) to efficiently approximate target aggregation and the Data-Diverse Drafts (DDD) strategy to enhance draft diversity for better target approximation. Extensive experiments on GSM8K demonstrate that our method reduces the estimation error (RMSE) by up to 37% compared to baselines. Crucially, our approach achieves Hallucination Detection performance competitive with heavy perturbation-based methods like TokUR while incurring negligible inference costs, offering a practical solution for uncertainty-aware LLM deployment.

翻译：量化大型语言模型（LLMs）的不确定性对于缓解幻觉问题及实现其在安全关键任务中的风险感知部署至关重要。然而，通过深度集成方法估计认知不确定性（EU）在现代模型规模下计算成本过高。我们提出一个框架，利用小型草稿模型高效估计词元级EU，从而绕过全规模集成的需求。该方法以偏差-方差分解为理论基础，通过草稿模型间的Jensen-Shannon散度（方差代理）以及草稿混合模型与目标模型间的KL散度（偏差代理）来近似EU。为进一步确保准确性且不引入显著开销，我们提出在线随机蒸馏（OSD）以高效近似目标聚合，并采用数据多样性草稿（DDD）策略增强草稿多样性以优化目标近似。在GSM8K数据集上的大量实验表明，相较于基线方法，我们的方法将估计误差（RMSE）降低了高达37%。关键的是，本方法在幻觉检测任务上取得了与TokUR等基于重度扰动的方法相竞争的性能，同时仅产生可忽略的推理开销，为不确定性感知的LLM部署提供了实用解决方案。