Large Language Models (LLMs) such as ChatGPT and LlaMA are advancing rapidly in generative Artificial Intelligence (AI), but their immense size poses significant challenges, such as huge training and inference costs, substantial energy demands, and limitations for on-site deployment. Traditional compression methods such as pruning, distillation, and low-rank approximation focus on reducing the effective number of neurons in the network, while quantization focuses on reducing the numerical precision of individual weights to reduce the model size while keeping the number of neurons fixed. While these compression methods have been relatively successful in practice, there's no compelling reason to believe that truncating the number of neurons is an optimal strategy. In this context, this paper introduces CompactifAI, an innovative LLM compression approach using quantum-inspired Tensor Networks that focuses on the model's correlation space instead, allowing for a more controlled, refined and interpretable model compression. Our method is versatile and can be implemented with - or on top of - other compression techniques. As a benchmark, we demonstrate that CompactifAI alone enables compression of the LlaMA-2 7B model to only $30\%$ of its original size while recovering over $90\%$ of the original accuracy after a brief distributed retraining.

翻译：大型语言模型（如ChatGPT和LlaMA）在生成式人工智能领域发展迅猛，但其庞大的规模带来了巨大挑战，例如高昂的训练和推理成本、巨大的能源需求以及现场部署的局限性。传统的压缩方法（如剪枝、蒸馏和低秩近似）侧重于减少网络中的有效神经元数量，而量化则侧重于降低单个权重的数值精度，在保持神经元数量不变的同时减小模型规模。尽管这些压缩方法在实践中取得了相对成功，但并无充分理由认为截断神经元数量是最优策略。在此背景下，本文提出CompactifAI，一种创新的LLM压缩方法，采用量子启发的张量网络，转而聚焦于模型的相关性空间，从而实现了更具可控性、精细化和可解释性的模型压缩。我们的方法具有通用性，可与其他压缩技术结合使用或在其基础上实现。作为基准测试，我们证明仅凭CompactifAI即可将LlaMA-2 7B模型压缩至其原始规模的30%，同时经过短暂分布式重新训练后恢复超过90%的原始精度。