Natural language generation is one of the most impactful fields in NLP, and recent years have witnessed its evolution brought about by large language models (LLMs). As the key instrument for writing assistance applications, they are generally prone to replicating or extending offensive content provided in the input. In low-resource data regime, they can also lead to repetitive outputs (Holtzman et al., 2019) [1]. Usually, offensive content and repetitions are mitigated with post-hoc methods, including n-gram level blocklists, top-k and nucleus sampling. In this paper, we introduce a combination of exact and non-exact repetition suppression using token and sequence level unlikelihood loss, repetition penalty during training, inference, and post-processing respectively. We further explore multi-level unlikelihood loss to the extent that it endows the model with abilities to avoid generating offensive words and phrases from the beginning. Finally, with comprehensive experiments, we demonstrate that our proposed methods work exceptionally in controlling the repetition and content quality of LLM outputs.

翻译：自然语言生成是自然语言处理领域最具影响力的方向之一，近年来大语言模型（LLMs）的发展推动了该领域的变革。作为写作辅助应用的核心工具，这类模型普遍存在复制或扩展输入中攻击性内容的问题。在低资源数据场景下，它们还容易产生重复输出（Holtzman等，2019）[1]。通常，攻击性内容和重复现象通过事后方法进行缓解，包括n-gram级黑名单、top-k采样及核采样。本文提出了一种结合精确与非精确重复抑制的策略，分别通过令牌级和序列级的不似然损失、训练及推理阶段的重复惩罚以及后处理技术实现。我们进一步探索了多层级不似然损失，使其能够从根源上赋予模型避免生成攻击性词汇和短语的能力。最终，通过全面实验证明，所提方法在控制LLM输出的重复性与内容质量方面表现出色。