Compressing neural networks is a key step when deploying models for real-time or embedded applications. Factorizing the model's matrices using low-rank approximations is a promising method for achieving compression. While it is possible to set the rank before training, this approach is neither flexible nor optimal. In this work, we propose a post-training rank-selection method called Rank-Tuning that selects a different rank for each matrix. Used in combination with training adaptations, our method achieves high compression rates with no or little performance degradation. Our numerical experiments on signal processing tasks show that we can compress recurrent neural networks up to 14x with at most 1.4% relative performance reduction.

翻译：压缩神经网络是部署模型用于实时或嵌入式应用的关键步骤。利用低秩近似对模型矩阵进行因式分解是一种具有前景的压缩方法。尽管可以在训练前设定秩，但这种方法既不灵活也非最优。本文提出一种名为Rank-Tuning的训练后秩选择方法，可为每个矩阵选择不同的秩。该方法与训练适应策略结合使用，能在几乎不降低或完全不降低性能的情况下实现高压缩率。我们在信号处理任务上的数值实验表明，循环神经网络压缩率最高可达14倍，同时相对性能损失不超过1.4%。