Inference-time algorithms are an emerging paradigm in which pre-trained models are used as subroutines to solve downstream tasks. Such algorithms have been proposed for tasks ranging from inverse problems and guided image generation to reasoning. However, the methods currently deployed in practice are heuristics with a variety of failure modes -- and we have very little understanding of when these heuristics can be efficiently improved. In this paper, we consider the task of sampling from a reward-tilted diffusion model -- that is, sampling from $p^{\star}(x) \propto p(x) \exp(r(x))$ -- given a reward function $r$ and pre-trained diffusion oracle for $p$. We provide a fine-grained analysis of the computational tractability of this task for quadratic rewards $r(x) = x^\top A x + b^\top x$. We show that linear-reward tilts are always efficiently sampleable -- a simple result that seems to have gone unnoticed in the literature. We use this as a building block, along with a conceptually new ingredient -- the Hubbard-Stratonovich transform -- to provide an efficient algorithm for sampling from low-rank positive-definite quadratic tilts, i.e. $r(x) = x^\top A x$ where $A$ is positive-definite and of rank $O(1)$. For negative-definite tilts, i.e. $r(x) = - x^\top A x$ where $A$ is positive-definite, we prove that the problem is intractable even if $A$ is of rank 1 (albeit with exponentially-large entries).

翻译：推理时算法是一种新兴范式，其中预训练模型被用作子程序来解决下游任务。此类算法已被提出用于从逆问题和引导图像生成到推理的各种任务。然而，目前实践中部署的方法是启发式的，存在多种失效模式——并且我们对这些启发式方法何时能被有效改进知之甚少。在本文中，我们考虑从奖励倾斜的扩散模型中采样的任务——即从 $p^{\star}(x) \propto p(x) \exp(r(x))$ 中采样——给定奖励函数 $r$ 和用于 $p$ 的预训练扩散预言机。我们针对二次奖励 $r(x) = x^\top A x + b^\top x$，对该任务的计算可处理性进行了细粒度分析。我们证明线性奖励倾斜总是可以有效采样的——这一简单结果似乎在文献中未被注意到。我们以此为基础，并结合一个概念上新的要素——哈伯德-斯特拉托诺维奇变换——为从低秩正定二次倾斜（即 $r(x) = x^\top A x$，其中 $A$ 是正定且秩为 $O(1)$）中采样提供了一种高效算法。对于负定倾斜（即 $r(x) = - x^\top A x$，其中 $A$ 是正定的），我们证明即使 $A$ 的秩为 1（尽管具有指数级大的条目），该问题也是难解的。