Sampling is a common strategy for generating text from probabilistic models, yet standard ancestral sampling often results in text that is incoherent or ungrammatical. To alleviate this issue, various modifications to a model's sampling distribution, such as nucleus or top-k sampling, have been introduced and are now ubiquitously used in language generation systems. We propose a unified framework for understanding these techniques, which we term sampling adapters. Sampling adapters often lead to qualitatively better text, which raises the question: From a formal perspective, how are they changing the (sub)word-level distributions of language generation models? And why do these local changes lead to higher-quality text? We argue that the shift they enforce can be viewed as a trade-off between precision and recall: while the model loses its ability to produce certain strings, its precision rate on desirable text increases. While this trade-off is not reflected in standard metrics of distribution quality (such as perplexity), we find that several precision-emphasizing measures indeed indicate that sampling adapters can lead to probability distributions more aligned with the true distribution. Further, these measures correlate with higher sequence-level quality scores, specifically, Mauve.

翻译：采样是从概率模型生成文本的常见策略，然而标准祖先采样常导致文本缺乏连贯性或语法规范。为解决该问题，研究者提出了多种对模型采样分布的修正方法（如核采样或top-k采样），这些方法现已广泛应用于语言生成系统。我们提出了一个统一框架来理解这些技术，并将其命名为"采样适配器"。采样适配器通常能生成质量更优的文本，这引发了一个问题：从形式化角度而言，它们如何改变语言生成模型的(子)词级分布？为何这些局部变化能产生更高质量的文本？我们认为，采样适配器施加的偏移可被视为精确率与召回率之间的权衡：模型虽会丧失生成特定字符串的能力，但其对优质文本的精确率得以提升。尽管这种权衡无法通过标准分布质量指标（如困惑度）反映，我们发现多个强调精确率的度量指标确实表明，采样适配器能引导概率分布更接近真实分布。此外，这些度量与更高的序列级质量评分（特别是Mauve指标）存在相关性。