We address LLM serving workloads where repeated requests share a common solution structure but differ in localized constraints, such as output schema, variable names, or numeric constants. Prior caching approaches typically reuse either full responses (semantic caching) or model-internal KV/prefix states, which are respectively brittle under partial changes or tightly coupled to specific backends. We present StepCache, a backend-agnostic step-level reuse layer that segments outputs into ordered steps, retrieves the best-matching cached request, verifies steps using lightweight task-aware checks, and regenerates only failing regions via selective patching. StepCache additionally supports strict structured-output enforcement for JSON, including single-step extraction, required-key constraints, and one-shot repair, as well as conservative skip-reuse fallbacks for semantic changes. For linear equations, StepCache promotes verification into correction via a bounded repair loop with a deterministic fallback that guarantees correctness when the backend model fails. In a CPU-only perturbation-heavy micro-benchmark on math and JSON variants, averaged over three seeds, StepCache reduces mean latency from 2.13 s to 0.67 s, median latency from 2.42 s to 0.01 s, and p95 latency from 3.38 s to 3.30 s. It also reduces total token usage from 36.1k to 27.3k and improves end-to-end correctness from 72.5% to 100% under task-specific checks and a stitched-output integrity check. Across requests, 79.7% take the reuse-only fast path, 5.4% require patching, and 14.9% trigger skip-reuse.

翻译：[translated abstract in Chinese] 我们解决大语言模型（LLM）服务负载中，重复请求共享相同解决方案结构但在局部约束（如输出模式、变量名或数值常量）上存在差异的问题。现有缓存方法通常复用完整响应（语义缓存）或模型内部的KV/前缀状态，前者在局部变更时脆弱易错，后者则与特定后端紧密耦合。我们提出StepCache——一种后端无关的步骤级复用层，将输出分割为有序步骤，检索最佳匹配的缓存请求，通过轻量级任务感知校验验证步骤，并仅通过选择性修补再生失败区域。StepCache还支持JSON的严格结构化输出强制，包括单步提取、必要键约束和单次修复，以及针对语义变更的保守跳过-回退策略。对于线性方程，StepCache通过带确定回退的有界修复循环将验证升级为修正，确保在后端模型失败时保证正确性。在仅含CPU的高扰动微基准测试（数学与JSON变体）中，基于三个随机种子取平均，StepCache将平均延迟从2.13秒降至0.67秒，中位数延迟从2.42秒降至0.01秒，p95延迟从3.38秒降至3.30秒。总令牌使用量从36.1k降至27.3k，在任务特定校验与拼接输出完整性校验下，端到端正确率从72.5%提升至100%。跨请求分析显示，79.7%走仅复用快速路径，5.4%需修补，14.9%触发跳过-回退。