Current language models decode text token by token according to probabilistic distribution, and determining the appropriate candidates for the next token is crucial to ensure generation quality. This study introduces adaptive decoding, a mechanism that empowers the language models to ascertain a sensible candidate set during the generation process dynamically. Specifically, we introduce an entropy-based metric called confidence and conceptualize determining the optimal candidate set as a confidence-increasing process. The rationality of including a token in the candidate set is assessed by leveraging the increment of confidence, enabling the model to determine the most suitable candidate set adaptively. The experimental results reveal that our method achieves higher MAUVE and diversity in story generation tasks and maintains certain coherence, underscoring its superiority over existing algorithms. The code is available at https://github.com/zwhong714/adaptive_decoding.

翻译：当前语言模型依据概率分布逐词解码文本，确定下一个词的合适候选集对确保生成质量至关重要。本研究提出自适应解码机制，该机制使语言模型能够在生成过程中动态确定合理的候选集。具体而言，我们引入了基于熵的置信度指标，并将最优候选集的确定概念化为置信度递增过程。通过利用置信度增量来评估将某个词纳入候选集的合理性，使模型能够自适应地确定最合适的候选集。实验结果表明，我们的方法在故事生成任务中实现了更高的MAUVE值和多样性，并保持了一定的连贯性，凸显了其相对于现有算法的优越性。代码已开源在https://github.com/zwhong714/adaptive_decoding。