Large language models (LLMs) have demonstrated remarkable capabilities across diverse tasks. However, the truthfulness of their outputs is not guaranteed, and their tendency toward overconfidence further limits reliability. Uncertainty quantification offers a promising way to identify potentially unreliable outputs, but most existing methods rely on repeated sampling or auxiliary models, introducing substantial computational overhead. To address these limitations, we propose Semantic Token Clustering (STC), an efficient uncertainty quantification method that leverages the semantic information inherently encoded in LLMs. Specifically, we group tokens into semantically consistent clusters using embedding clustering and prefix matching, and quantify uncertainty based on the probability mass aggregated over the corresponding semantic cluster. Our approach requires only a single generation and does not depend on auxiliary models. Experimental results show that STC achieves performance comparable to state-of-the-art baselines while substantially reducing computational overhead.

翻译：大语言模型（LLMs）已在多种任务中展现出卓越能力。然而，其输出真实性无法保证，且过度自信的倾向进一步限制了可靠性。不确定性量化为识别潜在不可靠输出提供了有效途径，但现有方法多依赖重复采样或辅助模型，引入大量计算开销。为解决上述局限，我们提出语义标记聚类（STC）——一种利用LLMs固有语义信息的高效不确定性量化方法。具体而言，通过嵌入聚类与前缀匹配将标记分组为语义一致簇，并基于对应语义簇上聚合的概率质量量化不确定性。本方法仅需单次生成且无需依赖辅助模型。实验表明，STC在显著降低计算开销的同时，性能可比肩最先进的基线方法。