In Text-to-SQL tasks, existing LLM-based methods often include extensive database schemas in prompts, leading to long context lengths and increased prefilling latency. While user queries typically focus on recurrent table sets-offering an opportunity for KV cache sharing across queries-current inference engines, such as SGLang and vLLM, generate redundant prefix cache copies when processing user queries with varying table orders. To address this inefficiency, we propose precomputing table representations as KV caches offline and querying the required ones online. A key aspect of our approach is the computation of table caches while preserving primary foreign key relationships between tables. Additionally, we construct a Table Trie structure to facilitate efficient KV cache lookups during inference. To enhance cache performance, we introduce a cache management system with a query reranking strategy to improve cache hit rates and a computation loading pipeline for parallelizing model inference and cache loading. Experimental results show that our proposed TableCache achieves up to a 3.62x speedup in Time to First Token (TTFT) with negligible performance degradation.

翻译：在Text-to-SQL任务中，现有基于大语言模型的方法通常会在提示中包含完整的数据库模式，导致上下文长度过长并增加预填充延迟。尽管用户查询通常聚焦于重复出现的表集合——这为跨查询共享KV缓存提供了机会——但当前推理引擎（如SGLang和vLLM）在处理具有不同表顺序的用户查询时，会生成冗余的前缀缓存副本。为解决这一效率问题，我们提出离线预计算表表示作为KV缓存，并在在线阶段查询所需缓存。我们方法的一个关键方面是在计算表缓存时保持表间的主外键关系。此外，我们构建了Table Trie数据结构以支持推理期间高效的KV缓存查找。为提升缓存性能，我们引入了一个缓存管理系统，其中包含用于提高缓存命中率的查询重排序策略，以及用于并行化模型推理与缓存加载的计算加载流水线。实验结果表明，我们提出的TableCache在首词生成时间上最高可实现3.62倍的加速，且性能下降可忽略不计。