As Large Language Models (LLMs) are deployed more widely, customization with respect to vocabulary, style, and character becomes more important. In this work, we introduce model arithmetic, a novel inference framework for composing and biasing LLMs without the need for model (re)training or highly specific datasets. In addition, the framework allows for more precise control of generated text than direct prompting and prior controlled text generation (CTG) techniques. Using model arithmetic, we can express prior CTG techniques as simple formulas and naturally extend them to new and more effective formulations. Further, we show that speculative sampling, a technique for efficient LLM sampling, extends to our setting. This enables highly efficient text generation with multiple composed models with only marginal overhead over a single model. Our empirical evaluation demonstrates that model arithmetic allows fine-grained control of generated text while outperforming state-of-the-art on the task of toxicity reduction. We release an open source easy-to-use implementation of our framework at https://github.com/eth-sri/language-model-arithmetic.

翻译：随着大语言模型（LLMs）的广泛部署，在词汇、风格和角色特征方面的个性化定制变得愈发重要。本文提出模型算术（model arithmetic）这一新型推理框架，可在无需模型（重新）训练或特定数据集的情况下，实现对LLMs的组合与偏置。该框架相比直接提示和先前受控文本生成（CTG）技术，能够更精准地控制生成文本。通过模型算术，我们可将已有CTG技术表达为简洁公式，并自然扩展至更新更有效的公式体系。此外，研究表明推测性采样（speculative sampling）这一高效LLM采样技术可扩展至本场景，使多模型组合下的文本生成效率接近单模型水平。实证评估表明，模型算术在实现细粒度文本控制的同时，在毒性降低任务上超越了现有最优方法。我们已在https://github.com/eth-sri/language-model-arithmetic 开源了该框架的易用实现。