Can a diffusion model trained on bedrooms recover human faces? Diffusion models are widely used as priors for inverse problems, but standard approaches usually assume a high-fidelity model trained on data that closely match the unknown signal. In practice, one often must use a mismatched or low-fidelity diffusion prior. Surprisingly, these weak priors often perform nearly as well as full-strength, in-domain baselines. We study when and why inverse solvers are robust to weak diffusion priors. Through extensive experiments, we find that weak priors succeed when measurements are highly informative (e.g., many observed pixels), and we identify regimes where they fail. Our theory, based on Bayesian consistency, gives conditions under which high-dimensional measurements make the posterior concentrate near the true signal. These results provide a principled justification on when weak diffusion priors can be used reliably.
翻译:在卧室图像上训练的扩散模型能否恢复人脸?扩散模型被广泛用作逆问题的先验模型,但标准方法通常假设使用与未知信号高度匹配的数据训练得到的高保真模型。实践中,研究者往往不得不使用不匹配或低保真度的扩散先验。令人惊讶的是,这些弱先验的表现常与完整强度的领域内基线方法相当。我们研究了逆问题求解器何时以及为何对弱扩散先验具有鲁棒性。通过大量实验,我们发现当测量信息高度丰富时(例如观测到大量像素),弱先验能够取得成功,并识别了其失效的机制。我们基于贝叶斯一致性的理论给出了高维测量使后验分布集中于真实信号附近的条件。这些结果为弱扩散先验何时能被可靠使用提供了理论依据。