Tools have become a mainstay of LLMs, allowing them to retrieve knowledge not in their weights, to perform tasks on the web, and even to control robots. However, most ontologies and surveys of tool-use have assumed the core challenge for LLMs is choosing the tool. Instead, we introduce a framework for tools more broadly which guides us to explore a model's ability to detect "silent" tool errors, and reflect on how to plan. This more directly aligns with the increasingly popular use of models as tools. We provide an initial approach to failure recovery with promising results both on a controlled calculator setting and embodied agent planning.

翻译：工具已成为大型语言模型（LLM）的核心组件，使其能够检索权重中未包含的知识、在网络上执行任务，甚至控制机器人。然而，大多数关于工具使用的本体论和综述研究都假定LLM面临的核心挑战在于工具选择。与此不同，我们提出了一个更广义的工具框架，该框架引导我们探索模型检测"静默"工具错误的能力，并反思如何进行规划。这更直接地契合了日益流行的将模型本身作为工具使用的趋势。我们提出了一种初步的故障恢复方法，在受控计算器环境和具身智能体规划任务中均取得了具有前景的结果。