Recent work has shown that energy-based language modeling is an effective framework for controllable text generation because it enables flexible integration of arbitrary discriminators. However, because energy-based LMs are globally normalized, approximate techniques like Metropolis-Hastings (MH) are required for inference. Past work has largely explored simple proposal distributions that modify a single token at a time, like in Gibbs sampling. In this paper, we develop a novel MH sampler that, in contrast, proposes re-writes of the entire sequence in each step via iterative prompting of a large language model. Our new sampler (a) allows for more efficient and accurate sampling from a target distribution and (b) allows generation length to be determined through the sampling procedure rather than fixed in advance, as past work has required. We perform experiments on two controlled generation tasks, showing both downstream performance gains and more accurate target distribution sampling in comparison with single-token proposal techniques.

翻译：近期研究显示，基于能量的语言建模作为可控文本生成的有效框架，能够灵活集成任意判别器。然而，由于能量语言模型需全局归一化，推理过程必须采用梅特罗波利斯-黑斯廷斯等近似技术。以往研究主要探索类似吉布斯采样的简单提议分布，即每次仅修改单个词符。本文提出一种新型MH采样器，其通过迭代提示大语言模型，在每一步对完整序列提出重写方案。该新型采样器具备以下特性：(a) 能够从目标分布中实现更高效、更精确的采样；(b) 采样过程中可自主确定生成长度，无需像先前研究那样预先固定。我们在两个可控生成任务上进行实验，结果表明：相较于单词符提议技术，本方法在下游性能与目标分布采样的准确性上均取得提升。