Large language models (LLMs) have shown promising capabilities in using external tools to solve complex problems. However, existing approaches either involve fine-tuning on tool demonstrations, which do not generalize to new tools without additional training, or providing tool documentation in context, limiting the number of tools. Both approaches often generate syntactically invalid tool calls. In this paper, we propose ToolDec, a finite-state machine-guided decoding algorithm for tool-augmented LLMs. ToolDec eliminates tool-related errors for any tool-augmented LLMs by ensuring valid tool names and type-conforming arguments. Furthermore, ToolDec enables LLM to effectively select tools using only the information contained in their names, with no need for fine-tuning or in-context documentation. We evaluated multiple prior methods and their ToolDec-enhanced versions on a variety of tasks involving tools like math functions, knowledge graph relations, and complex real-world RESTful APIs. Our experiments show that ToolDec reduces syntactic errors to zero, consequently achieving significantly better performance and as much as a 2x speedup. We also show that ToolDec achieves superior generalization performance on unseen tools, performing up to 8x better than the baselines.

翻译：大语言模型（LLM）在使用外部工具解决复杂问题方面展现出有前景的能力。然而，现有方法要么需要对工具示例进行微调（无法在无额外训练的情况下泛化到新工具），要么在上下文中提供工具文档（限制了工具数量）。这两种方法经常生成语法无效的工具调用。本文提出ToolDec——一种基于有限状态机引导的工具增强型LLM解码算法。ToolDec通过确保工具名称有效且参数类型合规，消除了任何工具增强型LLM的工具相关错误。此外，ToolDec仅利用工具名称包含的信息即可使LLM有效选择工具，无需微调或上下文文档。我们在涉及数学函数、知识图谱关系以及复杂真实世界RESTful API等工具的多种任务上评估了多种先前方法及其ToolDec增强版本。实验表明，ToolDec将语法错误降至零，从而显著提升性能，并实现高达2倍的加速。我们还证明，ToolDec在未见过的工具上实现了卓越的泛化性能，性能比基线方法高出8倍。