This paper concerns the structure of learned representations in text-guided generative models, focusing on score-based models. A key property of such models is that they can compose disparate concepts in a `disentangled' manner. This suggests these models have internal representations that encode concepts in a `disentangled' manner. Here, we focus on the idea that concepts are encoded as subspaces of some representation space. We formalize what this means, show there's a natural choice for the representation, and develop a simple method for identifying the part of the representation corresponding to a given concept. In particular, this allows us to manipulate the concepts expressed by the model through algebraic manipulation of the representation. We demonstrate the idea with examples using Stable Diffusion. Code in https://github.com/zihao12/concept-algebra-code

翻译：本文关注文本引导生成模型中学习到的表示结构，重点关注基于分数的模型。这类模型的一个关键特性是能够以“解耦”方式组合不同概念。这表明这些模型内部表示以“解耦”方式编码概念。本文聚焦于概念被编码为某个表示空间子空间这一思想。我们形式化定义了其含义，展示了表示的合理自然选择，并开发了一种识别给定概念对应表示部分的简单方法。特别地，这使得我们能够通过对表示的代数操作来操控模型表达的概念。我们通过Stable Diffusion示例演示该思想。代码见：https://github.com/zihao12/concept-algebra-code