Autoregressive Large Language Models (e.g., LLaMa, GPTs) are omnipresent achieving remarkable success in language understanding and generation. However, such impressive capability typically comes with a substantial model size, which presents significant challenges for autoregressive token-by-token generation. To mitigate computation overload incurred during generation, several early-exit and layer-dropping strategies have been proposed. Despite some promising success due to the redundancy across LLMs layers on metrics like Rough-L/BLUE, our careful knowledge-intensive evaluation unveils issues such as generation collapse, hallucination of wrong facts, and noticeable performance drop even at the trivial exit ratio of 10-15% of layers. We attribute these errors primarily to ineffective handling of the KV cache through state copying during early-exit. In this work, we observed the saturation of computationally expensive feed-forward blocks of LLM layers and proposed FFN-SkipLLM, which is a novel fine-grained skip strategy of autoregressive LLMs. More specifically, FFN-SkipLLM is an input-adaptive feed-forward skipping strategy that can skip 25-30% of FFN blocks of LLMs with marginal change in performance on knowledge-intensive generation tasks without any requirement to handle KV cache. Our extensive experiments and ablation across benchmarks like MT-Bench, Factoid-QA, and variable-length text summarization illustrate how our simple and ease-at-use method can facilitate faster autoregressive decoding.

翻译：自回归大语言模型（如LLaMa、GPTs）在语言理解与生成领域取得了显著成功，已无处不在。然而，这种强大能力通常伴随着庞大的模型规模，给逐词自回归生成带来了巨大挑战。为减轻生成过程中的计算负担，学者们提出了多种早期退出和层级跳过策略。尽管基于Rough-L/BLUE等指标发现LLM层间存在冗余现象，并取得了一些有前景的成果，但我们的严谨知识密集型评估揭示了以下问题：即使仅跳过10-15%的层，也会出现生成崩溃、错误事实幻觉及显著性能下降。我们将这些错误主要归因于早期退出过程中通过状态复制处理KV缓存的方式效率低下。本文观察到LLM层中计算密集的前馈模块存在饱和现象，进而提出FFN-SkipLLM——一种新颖的自回归LLM细粒度跳过策略。具体而言，FFN-SkipLLM是一种输入自适应前馈跳过策略，可在无需处理KV缓存的情况下，跳过LLM中25-30%的前馈模块，同时在知识密集型生成任务上仅产生微小性能变化。我们在MT-Bench、Factoid-QA及变长文本摘要等基准测试上开展的广泛实验与消融研究，展示了我们这种简单易用的方法如何加速自回归解码过程。