Current language models decode text token by token according to probabilistic distribution, and determining the appropriate candidates for the next token is crucial to ensure generation quality. This study introduces adaptive decoding, a mechanism that dynamically empowers language models to ascertain a sensible candidate set during generation. Specifically, we introduce an entropy-based metric called confidence and conceptualize determining the optimal candidate set as a confidence-increasing process. The rationality of including a token in the candidate set is assessed by leveraging the increment of confidence. Experimental results reveal that our method balances diversity and coherence well. The human evaluation shows that our method can generate human-preferred text. Additionally, our method can potentially improve the reasoning ability of language models.

翻译：当前语言模型根据概率分布逐词元解码文本，确定下一词元的适当候选集对确保生成质量至关重要。本研究提出自适应解码机制，该机制能够动态增强语言模型在生成过程中确定合理候选集的能力。具体而言，我们引入一种基于熵的度量指标——置信度，并将确定最优候选集的过程概念化为置信度提升过程。通过利用置信度的增量来评估将特定词元纳入候选集的合理性。实验结果表明，我们的方法能良好平衡多样性与连贯性。人工评估显示该方法能够生成人类更偏好的文本。此外，本方法还有望提升语言模型的推理能力。