Transformer-based Large Language Models (LLMs) have shown exceptional language generation capabilities in response to text-based prompts. However, controlling the direction of generation via textual prompts has been challenging, especially with smaller models. In this work, we explore the use of Prompt Tuning to achieve controlled language generation. Generated text is steered using prompt embeddings, which are trained using a small language model, used as a discriminator. Moreover, we demonstrate that these prompt embeddings can be trained with a very small dataset, with as low as a few hundred training examples. Our method thus offers a data and parameter efficient solution towards controlling language model outputs. We carry out extensive evaluation on four datasets: SST-5 and Yelp (sentiment analysis), GYAFC (formality) and JIGSAW (toxic language). Finally, we demonstrate the efficacy of our method towards mitigating harmful, toxic, and biased text generated by language models.

翻译：基于Transformer的大语言模型（LLMs）在响应文本提示时展现了卓越的语言生成能力。然而，通过文本提示控制生成方向仍具挑战性，尤其是对于较小规模的模型。本研究探索了利用提示调优实现受控语言生成的方法。生成文本通过提示嵌入进行引导，该嵌入由作为判别器的小型语言模型训练得到。此外，我们证明这些提示嵌入仅需极少量数据集（低至数百个训练样本）即可完成训练。因此，我们的方法为控制语言模型输出提供了数据高效且参数高效的解决方案。我们在四个数据集上进行了全面评估：SST-5和Yelp（情感分析）、GYAFC（正式程度）以及JIGSAW（有害语言）。最后，我们展示了该方法在缓解语言模型生成有害、有毒及偏见文本方面的有效性。