We investigate whether large language models (LLMs) can predict whether they will succeed on a given task and whether their predictions improve as they progress through multi-step tasks. We also investigate whether LLMs can learn from in-context experiences to make better decisions about whether to pursue a task in scenarios where failure is costly. All LLMs we tested are overconfident, but most predict their success with better-than-random discriminatory power. We find that newer and larger LLMs generally do not have greater discriminatory power, though Claude models do show such a trend. On multi-step agentic tasks, the overconfidence of several frontier LLMs worsens as they progress through the tasks, and reasoning LLMs perform comparably to or worse than non-reasoning LLMs. With in-context experiences of failure, some but not all LLMs reduce their overconfidence leading to significantly improved decision making, while others do not. Interestingly, all LLMs' decisions are approximately rational given their estimated probabilities of success, yet their overly-optimistic estimates result in poor decision making. These results suggest that current LLM agents are hindered by their lack of awareness of their own capabilities. We discuss the implications of LLMs' awareness of their capabilities for AI misuse and misalignment risks.

翻译：本研究探讨大型语言模型（LLMs）能否预测自身在给定任务上的成功率，以及其预测能力是否在多步骤任务推进过程中得到改善。同时，我们研究了在失败代价高昂的场景中，LLMs能否通过上下文经验学习，从而对是否执行任务做出更优决策。所有被测LLMs均表现出过度自信倾向，但多数模型展现出了优于随机水平的成功率判别能力。研究发现，更新、更大规模的LLMs通常并未具备更强的判别能力，尽管Claude模型系列显示出此类趋势。在多步骤智能体任务中，若干前沿LLMs的过度自信程度随任务推进而加剧，且具备推理能力的LLMs表现与非推理型LLMs相当或更差。在引入失败上下文经验后，部分（但非全部）LLMs降低了过度自信，决策能力得到显著改善。值得注意的是，所有LLMs的决策在给定其预估成功率的情况下均近似理性，但其过度乐观的估计仍导致决策质量低下。这些结果表明，当前LLM智能体因缺乏对自身能力的认知而受限。本文进一步探讨了LLMs能力认知对AI滥用与错位风险的影响。