Large language models (LLMs) are often augmented with tools to solve complex tasks. By generating code snippets and executing them through task-specific Application Programming Interfaces (APIs), they can offload certain functions to dedicated external modules, such as image encoding and performing calculations. However, most existing approaches to augment LLMs with tools are constrained by general-purpose APIs and lack the flexibility for tailoring them to specific tasks. In this work, we present CRAFT, a general tool creation and retrieval framework for LLMs. It creates toolsets specifically curated for the tasks and equips LLMs with a component that retrieves tools from these sets to enhance their capability to solve complex tasks. For each task, we collect specific code solutions by prompting GPT-4 to solve the training examples. Following a validation step ensuring the correctness, these solutions are abstracted into code snippets to enhance reusability, and deduplicated for higher quality. At inference time, the language model retrieves snippets from the toolsets and then executes them or generates the output conditioning on the retrieved snippets. Our method is designed to be flexible and offers a plug-and-play approach to adapt off-the-shelf LLMs to unseen domains and modalities, without any finetuning. Experiments on vision-language, tabular processing, and mathematical reasoning tasks show that our approach achieves substantial improvements compared to strong baselines. In addition, our in-depth analysis reveals that: (1) consistent performance improvement can be achieved by scaling up the number of tools and the capability of the backbone models; (2) each component of our approach contributes to the performance gains; (3) the created tools are well-structured and reliable with low complexity and atomicity. The code is available at \url{https://github.com/lifan-yuan/CRAFT}.

翻译：大型语言模型（LLM）常通过工具增强以处理复杂任务。通过生成代码片段并通过任务特定应用程序接口（API）执行这些代码，LLM可将图像编码、数值计算等功能外包给专用外部模块。然而，现有的大多数LLM工具增强方法受限于通用型API，缺乏针对特定任务进行定制的灵活性。本文提出CRAFT——一种面向LLM的通用工具创建与检索框架。该框架为任务构建专用工具集，并赋予LLM从工具集中检索工具的组件，从而增强其解决复杂任务的能力。针对每个任务，我们通过引导GPT-4求解训练样例来收集特定代码解决方案。经正确性验证后，这些解决方案被抽象为可复用的代码片段，并通过去重提升质量。在推理阶段，语言模型从工具集中检索代码片段，随后执行这些片段或基于检索结果生成输出。该方法具备灵活性与即插即用特性，无需微调即可使现成LLM适配未知领域与模态。在视觉-语言、表格处理及数学推理任务上的实验表明，本方法相较于强基线模型取得了显著改进。此外，深入分析揭示：(1) 通过扩展工具数量与骨干模型能力可取得一致的性能提升；(2) 本方法的每个组件均对性能增益有所贡献；(3) 所创建工具具有低复杂度与原子性，结构清晰且可靠。代码开源于 \url{https://github.com/lifan-yuan/CRAFT}。