Tool learning aims to extend the capabilities of large language models (LLMs) with external tools. A major challenge in tool learning is how to support a large number of tools, including unseen tools. To address this challenge, previous studies have proposed retrieving suitable tools for the LLM based on the user query. However, previously proposed methods do not consider the differences between seen and unseen tools, nor do they take the hierarchy of the tool library into account, which may lead to suboptimal performance for tool retrieval. Therefore, to address the aforementioned issues, we propose ToolRerank, an adaptive and hierarchy-aware reranking method for tool retrieval to further refine the retrieval results. Specifically, our proposed ToolRerank includes Adaptive Truncation, which truncates the retrieval results related to seen and unseen tools at different positions, and Hierarchy-Aware Reranking, which makes retrieval results more concentrated for single-tool queries and more diverse for multi-tool queries. Experimental results show that ToolRerank can improve the quality of the retrieval results, leading to better execution results generated by the LLM.

翻译：摘要：工具学习旨在通过外部工具扩展大语言模型（LLMs）的能力。该领域的一个核心挑战是如何支持包括未见工具在内的大量工具。为解决此问题，先前研究提出了基于用户查询为LLM检索合适工具的方法。然而，现有方法既未区分已见工具与未见工具之间的差异，也未考虑工具库的层级结构，这可能导致工具检索性能次优。为此，我们提出ToolRerank——一种面向工具检索的自适应层级感知重排序方法，以进一步优化检索结果。具体而言，本文方法包含自适应截断（Adaptive Truncation）与层级感知重排序（Hierarchy-Aware Reranking）两个模块：前者对已见及未见工具的相关检索结果在不同位置进行截断；后者则使单工具查询的检索结果更聚焦，多工具查询的结果更多样化。实验结果表明，ToolRerank能显著提升检索结果质量，从而让LLM生成更优的执行结果。