While Large Language Models (LLMs) have demonstrated enhanced capabilities in function-calling, these advancements primarily rely on accessing the functions' responses. This methodology is practical for simpler APIs but faces scalability issues with irreversible APIs that significantly impact the system, such as a database deletion API. Similarly, processes requiring extensive time for each API call and those necessitating forward planning, like automated action pipelines, present complex challenges. Furthermore, scenarios often arise where a generalized approach is needed because algorithms lack direct access to the specific implementations of these functions or secrets to use them. Traditional tool planning methods are inadequate in these cases, compelling the need to operate within black-box environments. Unlike their performance in tool manipulation, LLMs excel in black-box tasks, such as program synthesis. Therefore, we harness the program synthesis capabilities of LLMs to strategize tool usage in black-box settings, ensuring solutions are verified prior to implementation. We introduce TOPGUN, an ingeniously crafted approach leveraging program synthesis for black box tool planning. Accompanied by SwissNYF, a comprehensive suite that integrates black-box algorithms for planning and verification tasks, addressing the aforementioned challenges and enhancing the versatility and effectiveness of LLMs in complex API interactions. The public code for SwissNYF is available at https://github.com/iclr-dummy-user/SwissNYF.

翻译：尽管大语言模型（LLMs）在函数调用方面展现出增强能力，但这些进步主要依赖于访问函数的响应结果。该方法虽适用于简单API，但对于数据库删除类会严重影响系统的不可逆API存在扩展性问题。类似地，每次调用需消耗大量时间的API流程，以及需要前瞻性规划的自动化动作流水线等过程，也构成了复杂挑战。此外，当算法无法直接访问这些函数的具体实现细节或其使用秘诀时，往往需要普适性解决方案。传统工具规划方法在此类场景中难以奏效，这迫使我们需要在黑盒环境中运作。不同于在工具操控方面的表现，LLMs在程序合成等黑盒任务中展现出卓越能力。因此，我们利用LLMs的程序合成能力在黑盒场景中规划工具使用，确保解决方案在实施前得到验证。我们提出TOPGUN——一种巧妙利用程序合成实现黑盒工具规划的创新方法。配套发布的SwissNYF综合工具套件集成了用于规划与验证任务的黑盒算法，有效应对上述挑战，增强了LLMs在复杂API交互中的通用性与有效性。SwissNYF的公开代码可访问 https://github.com/iclr-dummy-user/SwissNYF 获取。