The dominant approaches for controlling language models achieve prominence in controlling high-level attributes (e.g. topic and sentiment). However, these methods often require condition-specific data or are computationally expensive. We propose a new simple guided decoding method, Gamma Sampling, which does not require any training data to achieve fine-grained controllable text generation while maintaining a fast generation speed. Gamma Sampling introduces attribute-related information (provided by humans or language models themselves) into the sampling process to guide language models to generate texts with desired attributes. Since no training is involved, Gamma Sampling can be easily applied to any language model for controllable text generation. Through experiments, we show that Gamma Sampling-steered GPT2-small (117M) outperforms baselines such as PPLM (345M) and CTRL (1.6B) in diversity, attribute relevance, and overall quality of generated samples.

翻译：控制语言模型的主要方法在控制高层次属性(如主题和情绪)方面占有突出地位,然而,这些方法往往需要特定条件的数据或计算成本昂贵。我们建议采用新的简单导解码方法Gamma抽样,这种方法不需要任何培训数据来实现细微的可控文本生成,同时保持快速生成速度。Gamma抽样将属性相关信息(由人或语言模型本身提供)引入取样过程,以指导语言模型生成具有理想属性的文本。由于不涉及培训,Gamma抽样可以很容易地适用于可控文本生成的任何语言模型。我们通过实验显示,Gamma抽样样本(GPLM)(345M)和CTRL(1.6B)在多样性、属性相关性和生成样本的总体质量方面优于PPLM(345M)和CTRL(1.6B)等基线。