Large language models~(LLMs) exhibit exceptional performance in language tasks, yet their auto-regressive inference is limited due to high computational requirements and is sub-optimal due to the exposure bias. Inspired by speculative decoding and contrastive decoding, we introduce Speculative Contrastive Decoding~(SCD), a straightforward yet powerful decoding approach that leverages predictions from smaller language models~(LMs) to achieve both decoding acceleration and quality improvement. Extensive evaluations and analyses on four diverse language tasks demonstrate the effectiveness of SCD, showing that decoding efficiency and quality can compatibly benefit from one smaller LM.

翻译：大语言模型在语言任务中展现出卓越性能，但其自回归推理因高计算需求而受到限制，且因暴露偏差而并非最优。受推测解码和对比解码的启发，我们提出推测对比解码（SCD），这是一种简洁而强大的解码方法，利用较小语言模型的预测来实现解码加速和质量提升。在四项不同语言任务上的广泛评估和分析证明了SCD的有效性，表明解码效率和质量可以从一个较小的语言模型中协同受益。