The ability to use tools is fundamental for large language model (LLM) agents. Given a task, existing systems use LLMs to plan and generate tool calls, which are executed by real-world tools to complete the task. However, tool calls are prone to errors because they are generated primarily from the intrinsic capabilities of LLMs. Moreover, while it is useful to let LLMs iteratively refine the tool-call sequence using execution results from real tools, this process can be expensive and may cause unsafe side effects. To improve LLM tool calls and address issues caused by using real tools for refinement, we introduce Gecko, a stateful simulation environment that provides informative feedback for refining LLM tool calls before real execution. Specifically, Gecko combines rules and LLMs to check the validity of tool names and arguments, synthesize schema-conforming and state-consistent responses, and judge task completion against the user objective. These three types of feedback allow LLMs to refine their tool calls in simulation, forming a simple yet effective test-time scaling method named GATS. On BFCLv3 and $τ^2$-bench, GATS consistently improves the performance of various LLMs.

翻译：工具使用能力是大语言模型（LLM）智能体的核心功能。针对给定任务，现有系统利用LLM规划并生成工具调用，由真实工具执行以完成任务。然而，由于工具调用主要依赖LLM的内在能力生成，极易出现错误。此外，虽然利用真实工具的执行结果让LLM迭代优化工具调用序列具有实用价值，但该过程成本高昂且可能引发不安全副作用。为改进LLM工具调用并解决真实工具优化带来的问题，我们提出Gecko——一种具有状态反馈的模拟环境，可在真实执行前为LLM工具调用优化提供信息性反馈。具体而言，Gecko结合规则与LLM来验证工具名称与参数的有效性，合成符合模式规范且保持状态一致的响应，并根据用户目标判定任务完成度。这三类反馈使LLM能够在模拟环境中优化工具调用，形成一种简单有效的测试时扩展方法GATS。在BFCLv3和$τ^2$-bench基准上，GATS持续提升多种LLM的性能。