Large language models (LLMs) are widely used in decision-making, but their reliability, especially in critical tasks like healthcare, is not well-established. Therefore, understanding how LLMs reason and make decisions is crucial for their safe deployment. This paper investigates how the uncertainty of responses generated by LLMs relates to the information provided in the input prompt. Leveraging the insight that LLMs learn to infer latent concepts during pretraining, we propose a prompt-response concept model that explains how LLMs generate responses and helps understand the relationship between prompts and response uncertainty. We show that the uncertainty decreases as the prompt's informativeness increases, similar to epistemic uncertainty. Our detailed experimental results on real datasets validate our proposed model.

翻译：大型语言模型（LLMs）已广泛应用于决策任务，但其可靠性——尤其在医疗等关键领域——尚未得到充分验证。因此，理解LLMs的推理与决策机制对其安全部署至关重要。本文研究LLMs生成响应的不确定性与输入提示所提供信息之间的关联。基于LLMs在预训练过程中学习推断潜在概念的机制，我们提出一种提示-响应概念模型，该模型既能解释LLMs生成响应的过程，又有助于理解提示与响应不确定性之间的关系。研究表明，随着提示信息量的增加，响应不确定性会相应降低，这种现象与认知不确定性具有相似性。我们在真实数据集上的详细实验结果验证了所提出模型的有效性。