This paper concerns the control of text-guided generative models, where a user provides a natural language prompt and the model generates samples based on this input. Prompting is intuitive, general, and flexible. However, there are significant limitations: prompting can fail in surprising ways, and it is often unclear how to find a prompt that will elicit some desired target behavior. A core difficulty for developing methods to overcome these issues is that failures are know-it-when-you-see-it -- it's hard to fix bugs if you can't state precisely what the model should have done! In this paper, we introduce a formalization of "what the user intended" in terms of latent concepts implicit to the data generating process that the model was trained on. This formalization allows us to identify some fundamental limitations of prompting. We then use the formalism to develop concept algebra to overcome these limitations. Concept algebra is a way of directly manipulating the concepts expressed in the output through algebraic operations on a suitably defined representation of input prompts. We give examples using concept algebra to overcome limitations of prompting, including concept transfer through arithmetic, and concept nullification through projection. Code available at https://github.com/zihao12/concept-algebra.

翻译：本文关注文本引导生成模型的控制问题，用户提供自然语言提示，模型基于该输入生成样本。提示操作直观、通用且灵活，但存在显著局限性：提示可能以出乎意料的方式失效，且通常难以找到能引发期望目标行为的提示。开发解决方案的核心困难在于，失效问题具有"见则识之"的特性——若无法精确描述模型本应执行的操作，修复缺陷便无从谈起。本文提出将"用户意图"形式化为数据生成过程中隐含的潜在概念，该形式化基于模型训练所依据的数据生成过程。这一形式化方法使我们得以揭示提示操作的部分根本局限性，并进一步开发概念代数来克服这些局限。概念代数通过对输入提示的适当表征执行代数运算，直接操纵输出中表达的概念。我们通过示例展示了概念代数如何克服提示操作的局限，包括通过算术运算实现概念迁移，以及通过投影运算实现概念消除。代码见 https://github.com/zihao12/concept-algebra。