Text-to-video diffusion models deliver impressive results but remain slow because of the sequential denoising of 3D latents. Existing approaches to speed up inference either require expensive model retraining or use heuristic-based step skipping, which struggles to maintain video quality as the number of denoising steps decreases. Our work, CHAI, aims to use cross-inference caching to reduce latency while maintaining video quality. We introduce Cache Attention as an effective method for attending to shared objects/scenes across cross-inference latents. This selective attention mechanism enables effective reuse of cached latents across semantically related prompts, yielding high cache hit rates. We show that it is possible to generate high-quality videos using Cache Attention with as few as 8 denoising steps. When integrated into the overall system, CHAI is 1.65x - 3.35x faster than baseline OpenSora 1.2 while maintaining video quality.

翻译：文本到视频扩散模型能够生成令人印象深刻的结果，但由于需要对三维潜在表示进行顺序去噪，其推理速度仍然较慢。现有的加速推理方法要么需要昂贵的模型重新训练，要么采用基于启发式的步骤跳过策略，这在减少去噪步骤数量时难以维持视频质量。我们的工作CHAI旨在利用跨推理缓存来降低延迟，同时保持视频质量。我们引入了缓存注意力作为一种有效方法，用于关注跨推理潜在表示之间共享的对象/场景。这种选择性注意力机制能够有效地在语义相关的提示之间重用缓存的潜在表示，从而实现较高的缓存命中率。我们证明，使用缓存注意力仅需8个去噪步骤即可生成高质量视频。当集成到完整系统中时，CHAI在保持视频质量的同时，比基线OpenSora 1.2快1.65倍至3.35倍。