Obtaining high-quality outputs from Large Language Models (LLMs) often depends upon the choice of a sampling-based decoding strategy to probabilistically choose the next token at each generation step. While a variety of such sampling methods have been proposed, their performance can be sensitive to the selection of hyperparameters which may require different settings depending upon the generation task and temperature configuration. In this work, we introduce $p$-less sampling: an information-theoretic approach to sampling which dynamically sets a truncation threshold at each decoding step based on the entire token probability distribution. Unlike existing methods, $p$-less sampling has no hyperparameters and consistently produces high-quality outputs as temperature increases. We provide theoretical perspectives on $p$-less sampling to ground our proposed method and conduct experiments to empirically validate its effectiveness across a range of math, logical reasoning, and creative writing tasks. Our results demonstrate how $p$-less sampling consistently outperforms existing sampling approaches while exhibiting much less degradation in text quality at higher temperature values. We further show how $p$-less achieves greater inference-time efficiency than alternative methods through lower average token sampling times and shorter generation lengths, without sacrificing accuracy. Finally, we provide analyses to highlight the benefits of $p$-less through qualitative examples, case studies, and diversity assessments. The code is available at https://github.com/ryttry/p-less .

翻译：从大语言模型（LLMs）中获取高质量输出通常依赖于基于采样的解码策略，该策略在每一生成步骤中以概率方式选择下一个词元。尽管已有多种此类采样方法被提出，但其性能可能对超参数的选择较为敏感，且这些参数可能需要根据生成任务和温度配置进行不同设置。本文提出无p采样：一种基于信息论的采样方法，该方法根据完整的词元概率分布在每个解码步骤动态设置截断阈值。与现有方法不同，无p采样无需超参数，且能在温度升高时持续产生高质量输出。我们为无p采样提供了理论视角以支撑所提方法，并通过实验在一系列数学、逻辑推理和创意写作任务中实证验证其有效性。结果表明，无p采样在保持更高温度下文本质量退化程度显著降低的同时，始终优于现有采样方法。我们进一步证明，无p采样通过更低的平均词元采样时间和更短的生成长度，实现了比替代方法更高的推理效率，且未牺牲准确性。最后，我们通过定性示例、案例研究和多样性评估的分析，凸显了无p采样的优势。代码发布于 https://github.com/ryttry/p-less 。