This research aims to accelerate the inference speed of large language models (LLMs) with billions of parameters. We propose \textbf{S}mart \textbf{P}arallel \textbf{A}uto-\textbf{C}orrect d\textbf{E}coding (SPACE), an innovative approach designed for achieving lossless acceleration of LLMs. By integrating semi-autoregressive inference and speculative decoding capabilities, SPACE uniquely enables autoregressive LLMs to parallelize token generation and verification. This is realized through a specialized semi-autoregressive supervised fine-tuning process that equips existing LLMs with the ability to simultaneously predict multiple tokens. Additionally, an auto-correct decoding algorithm facilitates the simultaneous generation and verification of token sequences within a single model invocation. Through extensive experiments on a range of LLMs, SPACE has demonstrated inference speedup ranging from 2.7x-4.0x on HumanEval-X while maintaining output quality.

翻译：本研究旨在加速具有数十亿参数的大语言模型（LLMs）的推理速度。我们提出了一种名为智能并行自动纠错解码（SPACE）的创新方法，旨在实现LLM的无损加速。通过融合半自回归推理与推测解码能力，SPACE独特地使自回归LLM能够并行化令牌生成与验证过程。这一目标通过专用的半自回归监督微调流程实现，该流程赋予现有LLM同时预测多个令牌的能力。此外，自动纠错解码算法可在单次模型调用中同时完成令牌序列的生成与验证。在多种LLM上的大量实验表明，SPACE在HumanEval-X基准上实现了2.7倍至4.0倍的推理加速，同时保持输出质量不变。