The Large language models (LLMs) have showcased superior capabilities in sophisticated tasks across various domains, stemming from basic question-answer (QA), they are nowadays used as decision assistants or explainers for unfamiliar content. However, they are not always correct due to the data sparsity in specific domain corpus, or the model's hallucination problems. Given this, how much should we trust the responses from LLMs? This paper presents a novel way to evaluate the uncertainty that captures the directional instability, by constructing a directional graph from entailment probabilities, and we innovatively conduct Random Walk Laplacian given the asymmetric property of a constructed directed graph, then the uncertainty is aggregated by the derived eigenvalues from the Laplacian process. We also provide a way to incorporate the existing work's semantics uncertainty with our proposed layer. Besides, this paper identifies the vagueness issues in the raw response set and proposes an augmentation approach to mitigate such a problem, we conducted extensive empirical experiments and demonstrated the superiority of our proposed solutions.

翻译：大语言模型（LLM）已在诸多领域的复杂任务中展现出卓越能力，从基础的问答（QA）出发，如今它们被用作决策助手或陌生内容的解释器。然而，由于特定领域语料库的数据稀疏性或模型的幻觉问题，其回答并非总是正确的。鉴于此，我们应多大程度上信任大语言模型的响应？本文提出了一种评估不确定性的新方法，该方法通过从蕴含概率构建方向图来捕捉方向不稳定性，并针对所构建有向图的非对称特性创新性地进行随机游走拉普拉斯运算，进而通过拉普拉斯过程导出的特征值聚合不确定性。我们还提供了一种将现有工作的语义不确定性与我们提出的层次相结合的方法。此外，本文识别了原始响应集中的模糊性问题，并提出了一种增强方法来缓解该问题。我们进行了广泛的实证实验，证明了所提出解决方案的优越性。