Large Language Models (LLMs) have acquired ubiquitous attention for their performances across diverse domains. Our study here searches through LLMs' cognitive abilities and confidence dynamics. We dive deep into understanding the alignment between their self-assessed confidence and actual performance. We exploit these models with diverse sets of questionnaires and real-world scenarios and extract how LLMs exhibit confidence in their responses. Our findings reveal intriguing instances where models demonstrate high confidence even when they answer incorrectly. This is reminiscent of the Dunning-Kruger effect observed in human psychology. In contrast, there are cases where models exhibit low confidence with correct answers revealing potential underestimation biases. Our results underscore the need for a deeper understanding of their cognitive processes. By examining the nuances of LLMs' self-assessment mechanism, this investigation provides noteworthy revelations that serve to advance the functionalities and broaden the potential applications of these formidable language models.

翻译：大型语言模型（LLMs）因在多个领域的卓越表现而获得广泛关注。本研究深入探索LLMs的认知能力与信心动态，旨在理解其自我评估信心与实际表现之间的一致性。我们通过多样化的问卷和真实场景测试这些模型，揭示LLMs如何在其回应中展现信心。研究结果发现了耐人寻味的现象：模型在给出错误答案时仍表现出高度自信，这与人类心理学中的邓宁-克鲁格效应（Dunning-Kruger effect）相似；相反，也存在模型在回答正确时信心较低的情况，暗示潜在的低估偏差。这些发现凸显了深入理解其认知过程的必要性。通过剖析LLMs自我评估机制的细微差异，本研究提供了具有启示意义的发现，有助于增强这些强大语言模型的功能并拓展其潜在应用。