As Large Language Models (LLMs) are deployed more widely, customization with respect to vocabulary, style and character becomes more important. In this work we introduce model arithmetic, a novel inference framework for composing and biasing LLMs without the need for model (re)training or highly specific datasets. In addition, the framework allows for more precise control of generated text than direct prompting and prior controlled text generation (CTG) techniques. Using model arithmetic, we can express prior CTG techniques as simple formulas and naturally extend them to new and more effective formulations. Further, we show that speculative sampling, a technique for efficient LLM sampling, extends to our setting. This enables highly efficient text generation with multiple composed models with only marginal overhead over a single model. Our empirical evaluation demonstrates that model arithmetic allows fine-grained control of generated text while outperforming state-of-the-art on the task of toxicity reduction.

翻译：随着大型语言模型（LLMs）部署日益广泛，针对词汇、风格和角色特征的定制化需求愈发重要。本文提出模型算术这一新型推理框架，无需模型（重新）训练或高度特定的数据集即可实现LLMs的组合与偏置。相较于直接提示和先前的受控文本生成（CTG）技术，该框架能对生成文本施加更精确的控制。通过模型算术，我们可将现有CTG技术表达为简洁公式，并自然扩展形成更有效的新公式。此外，我们证明高效LLM采样中的推测采样技术可适配至本框架。这使得多组合模型的高效文本生成成为可能，且相比单个模型仅产生边际开销。实证评估表明，模型算术在实现生成文本细粒度控制的同时，在毒性降低任务上显著超越现有最优方法。