Large language models (LLMs) deliver impressive performance but incur prohibitive memory and compute costs at deployment. Model pruning is an effective way to reduce these overheads, yet existing approaches face challenges: unstructured sparsity, where nonzeros can appear anywhere, preserves accuracy but yields irregular access patterns that prevent GPU acceleration, while semi-structured 2:4 sparsity is hardware-friendly but enforces a rigid 50% pattern that degrades model quality. To bridge this gap, we introduce PATCH, a hybrid sparsity framework that enables a continuous sparsity ratio between 0% and 50%. PATCH partitions weight matrices into tiles, assigning each tile to be either dense or 2:4 sparse via a learnable mask selection mechanism. This design provides fine-grained control over accuracy-acceleration tradeoffs and supports non-uniform sparsity across layers, leading to superior overall quality. Across models from 0.5B to 13B parameters, PATCH consistently narrows the gap to dense accuracy while delivering practical speedups. For instance, on LLaMA-2 7B with an A6000 GPU, PATCH achieves 1.18x-1.38x end-to-end speedup over dense baselines while improving accuracy by 0.37%-2.96% compared to the state-of-the-art 2:4 pruning method, MaskLLM.

翻译：大语言模型（LLMs）虽展现出卓越性能，但部署时会产生高昂的内存与计算开销。模型剪枝是降低这些开销的有效手段，但现有方法面临挑战：非结构化稀疏性允许非零元素任意分布，虽能保持精度却导致不规则访存模式从而阻碍GPU加速；而半结构化2：4稀疏性虽兼容硬件，但强制要求50%固定稀疏率，会降低模型质量。为弥合这一鸿沟，我们提出PATCH——一种支持0%至50%连续稀疏率的混合稀疏性框架。该框架将权重矩阵划分为多个瓦片，并通过可学习的掩码选择机制为每个瓦片分配密集或2：4稀疏模式。这种设计既能细粒度地调控精度-加速比权衡，又可实现跨层的非均匀稀疏性，从而获得更优的整体质量。在0.5B至13B参数规模的模型上，PATCH持续缩小与稠密模型的精度差距，同时提供实际加速效果。例如，在搭载A6000 GPU的LLaMA-2 7B模型上，PATCH相较于稠密基线实现了1.18倍至1.38倍的端到端加速，且相比最先进的2：4剪枝方法MaskLLM，精度提升了0.37%至2.96%。