As generative models become ubiquitous, there is a critical need for fine-grained control over the generation process. Yet, while controlled generation methods from prompting to fine-tuning proliferate, a fundamental question remains unanswered: are these models truly controllable in the first place? In this work, we provide a theoretical framework to formally answer this question. Framing human-model interaction as a control process, we propose a novel algorithm to estimate the controllable sets of models in a dialogue setting. Notably, we provide formal guarantees on the estimation error as a function of sample complexity: we derive probably-approximately correct bounds for controllable set estimates that are distribution-free, employ no assumptions except for output boundedness, and work for any black-box nonlinear control system (i.e., any generative model). We empirically demonstrate the theoretical framework on different tasks in controlling dialogue processes, for both language models and text-to-image generation. Our results show that model controllability is surprisingly fragile and highly dependent on the experimental setting. This highlights the need for rigorous controllability analysis, shifting the focus from simply attempting control to first understanding its fundamental limits.

翻译：随着生成模型变得无处不在，对生成过程进行细粒度控制的需求日益迫切。然而，尽管从提示到微调的各种受控生成方法不断涌现，一个根本性问题仍未得到解答：这些模型本身是否真正可控？在本工作中，我们提出了一个理论框架来形式化地回答这个问题。通过将人机交互建模为一个控制过程，我们提出了一种新颖的算法来估计对话场景中模型的可控集。值得注意的是，我们为估计误差提供了关于样本复杂度的形式化保证：我们推导了可控集估计的概率近似正确界，这些界是分布无关的，除了输出有界性外不依赖任何假设，并且适用于任何黑盒非线性控制系统（即任何生成模型）。我们在控制对话过程的不同任务上，针对语言模型和文本到图像生成，对理论框架进行了实证验证。我们的结果表明，模型的可控性出人意料地脆弱，并且高度依赖于实验设置。这凸显了进行严格可控性分析的必要性，将研究重点从单纯尝试控制转向首先理解其根本限制。