Tool agents--LLM-based systems that interact with external APIs--offer a way to execute real-world tasks. However, as tasks become increasingly complex, these agents struggle to identify and call the correct APIs in the proper order. To tackle this problem, we investigate converting API documentation into a structured API graph that captures API dependencies and leveraging it for multi-tool queries that require compositional API calls. To support this, we introduce In-N-Out, the first expert-annotated dataset of API graphs built from two real-world API benchmarks and their documentation. Using In-N-Out significantly improves performance on both tool retrieval and multi-tool query generation, nearly doubling that of LLMs using documentation alone. Moreover, graphs generated by models fine-tuned on In-N-Out close 90% of this gap, showing that our dataset helps models learn to comprehend API documentation and parameter relationships. Our findings highlight the promise of using explicit API graphs for tool agents and the utility of In-N-Out as a valuable resource. We release our dataset and code at https://github.com/holi-lab/In-N-Out-API-Graph.

翻译：工具代理——即基于大语言模型（LLM）并与外部API交互的系统——提供了一种执行现实世界任务的途径。然而，随着任务日益复杂，这些代理难以识别并以正确的顺序调用恰当的API。为解决此问题，我们研究将API文档转换为一种结构化API图以捕获API间的依赖关系，并利用该图处理需要组合式API调用的多工具查询。为此，我们提出了In-N-Out，这是首个基于两个真实世界API基准及其文档构建、并由专家标注的API图数据集。使用In-N-Out显著提升了工具检索和多工具查询生成的性能，其效果近乎是仅使用文档的LLM的两倍。此外，通过在In-N-Out上微调的模型所生成的图，能弥合此性能差距的90%，这表明我们的数据集有助于模型学习理解API文档及参数间关系。我们的研究结果凸显了使用显式API图对于工具代理的潜力，以及In-N-Out作为一种宝贵资源的实用性。我们已在 https://github.com/holi-lab/In-N-Out-API-Graph 发布数据集与代码。