Recent advances in Large Language Models (LLMs) have revolutionized web applications, enabling intelligent search, recommendation, and assistant services with natural language interfaces. Tool-calling extends LLMs with the ability to interact with external APIs, greatly enhancing their practical utility. While prior research has improved tool-calling performance by adopting traditional computer systems techniques, such as parallel and asynchronous execution, the challenge of redundant or repeated tool-calling requests remains largely unaddressed. Caching is a classic solution to this problem, but applying it to LLM tool-calling introduces new difficulties due to heterogeneous request semantics, dynamic workloads, and varying freshness requirements, which render conventional cache policies ineffective. To address these issues, we propose ToolCaching, an efficient feature-driven and adaptive caching framework for LLM tool-calling systems. ToolCaching systematically integrates semantic and system-level features to evaluate request cacheability and estimate caching value. At its core, the VAAC algorithm integrates bandit-based admission with value-driven, multi-factor eviction, jointly accounting for request frequency, recency, and caching value. Extensive experiments on synthetic and public tool-calling workloads demonstrate that ToolCaching with VAAC achieves up to 11% higher cache hit ratios and 34% lower latency compared to standard policies, effectively accelerating LLM tool-calling in practical applications.

翻译：近年来，大语言模型（LLM）的进展彻底改变了网络应用，使其能够通过自然语言接口提供智能搜索、推荐和助手服务。工具调用功能扩展了LLM与外部API交互的能力，极大增强了其实用性。尽管先前研究通过采用传统计算机系统技术（如并行与异步执行）提升了工具调用性能，但冗余或重复的工具调用请求问题仍未得到充分解决。缓存是解决该问题的经典方案，然而将其应用于LLM工具调用时，由于异构的请求语义、动态工作负载以及多变的新鲜度要求，传统缓存策略难以奏效。为应对这些挑战，本文提出ToolCaching——一个面向LLM工具调用系统的高效特征驱动自适应缓存框架。ToolCaching系统性地整合语义特征与系统级特征，以评估请求的可缓存性并估算缓存价值。其核心VAAC算法将基于多臂老虎机的准入机制与价值驱动的多因素淘汰策略相结合，综合考虑请求频率、时效性及缓存价值。在合成与公开工具调用工作负载上的大量实验表明，相较于标准策略，采用VAAC的ToolCaching可实现高达11%的缓存命中率提升与34%的延迟降低，在实际应用中有效加速了LLM工具调用。