It has always been an important yet challenging problem to control language models to avoid generating texts with undesirable attributes, such as toxic language and unnatural repetition. We introduce Click for controllable text generation, which needs no modification to the model architecture and facilitates out-of-the-box use of trained models. It employs a contrastive loss on sequence likelihood, which fundamentally decreases the generation probability of negative samples (i.e., generations with undesirable attributes). It also adopts a novel likelihood ranking-based strategy to construct contrastive samples from model generations. On the tasks of language detoxification, sentiment steering, and repetition reduction, we show that Click outperforms strong baselines of controllable text generation and demonstrate the superiority of Click's sample construction strategy.

翻译：摘要：控制语言模型避免生成包含不良属性（如毒性语言与非自然重复）的文本，始终是一项重要而富有挑战性的问题。我们提出用于可控文本生成的Click方法，该方法无需修改模型架构，即可实现预训练模型的即插即用。它采用序列似然上的对比损失，从根本上降低了负样本（即具有不良属性的生成内容）的生成概率。同时，该方法还基于似然排序策略，从模型生成结果中构造对比样本。在语言去毒化、情感操控与重复抑制任务中，我们证明Click在可控文本生成方面优于强基线方法，并展示了Click样本构造策略的优越性。