We present ROCKET, a training-free model compression method that achieves state-of-the-art performance in comparison with factorization, structured-sparsification and dynamic compression baselines. Operating under a global compression budget, ROCKET comprises two key innovations: First, it formulates layer-wise compression allocation as a multi-choice knapsack problem, selecting the optimal compression level for each layer to minimize total reconstruction error while adhering to a target model size. Second, it introduces a single-step sparse matrix factorization inspired by dictionary learning: using only a small calibration set, it sparsifies weight coefficients based on activation-weights sensitivity and then updates the dictionary in closed form via least squares bypassing iterative optimization, sparse coding, or backpropagation entirely. ROCKET consistently outperforms existing compression approaches across different model architectures at 20-50\% compression rates. Notably, it retains over 90\% of the original model's performance at 30\% compression without any fine-tuning. Moreover, when applying a light fine-tuning phase, recovery is substantially enhanced: for instance, compressing Qwen3-14B to an 8B-parameter model and healing it with just 30 million tokens yields performance nearly on par with the original Qwen3-8B. The code for ROCKET is at github.com/mts-ai/ROCKET/tree/main.

翻译：本文提出ROCKET，一种无需训练的模型压缩方法，与因子化、结构化稀疏化和动态压缩基线方法相比，其性能达到当前最优水平。在全局压缩预算约束下，ROCKET包含两项关键创新：首先，它将逐层压缩分配建模为多选择背包问题，通过为每一层选择最优压缩级别，在满足目标模型大小的同时最小化总体重构误差。其次，它引入了一种受字典学习启发的单步稀疏矩阵因子化方法：仅使用少量校准数据，该方法基于激活-权重敏感性对权重系数进行稀疏化，随后通过最小二乘法以闭式解更新字典，完全绕过了迭代优化、稀疏编码或反向传播过程。在20-50%的压缩率下，ROCKET在不同模型架构上均持续优于现有压缩方法。值得注意的是，在30%压缩率下无需任何微调即可保持原模型90%以上的性能。此外，当应用轻量微调阶段时，性能恢复得到显著增强：例如，将Qwen3-14B压缩为80亿参数模型后，仅用3000万词元进行恢复训练，其性能即可接近原始Qwen3-8B的水平。ROCKET的代码发布于github.com/mts-ai/ROCKET/tree/main。