Long-context inference in large language models is increasingly bottlenecked by the memory and compute cost of the KV-Cache. Low-rank factorization compresses KV projections by writing $W \approx A * B$, where A produces latent KV states and B can be absorbed into downstream weights. In modern RoPE-based LLMs, this absorption fails: RoPE forces latent KV states to be reconstructed to full dimension, reintroducing substantial memory and compute overhead. We propose RoPE-Aligned Pruning (RAP), which prunes entire RoPE-aligned column pairs to preserve RoPE's 2x2 rotation structure, restore B absorption, and eliminate reconstruction. Our evaluation on LLaMA-3-8B and Mistral-7B shows that RAP enables joint reduction of KV-Cache, attention parameters, and FLOPs by 20-30%, all at once, while maintaining strong accuracy. Notably, RAP reduces attention latency to 83% (prefill) and 77% (decode) of baseline.

翻译：大型语言模型的长上下文推理日益受到KV缓存内存与计算成本的瓶颈制约。低秩分解通过将$W \approx A * B$形式压缩KV投影矩阵，其中A生成潜在KV状态，B可吸收至下游权重。在现代基于RoPE的LLM中，这种吸收机制失效：RoPE迫使潜在KV状态重构至完整维度，重新引入显著的内存与计算开销。本文提出RoPE对齐剪枝（RAP），通过剪枝完整的RoPE对齐列对以保持RoPE的2x2旋转结构，恢复B吸收能力并消除重构需求。在LLaMA-3-8B和Mistral-7B上的实验表明，RAP能同步实现KV缓存、注意力参数与FLOPs的20-30%联合压缩，同时保持较强的准确性。值得注意的是，RAP将注意力延迟降低至基准线的83%（预填充阶段）和77%（解码阶段）。