Recent work on discrete generative priors, in the form of codebooks, has shown exciting performance for image reconstruction and restoration, as the discrete prior space spanned by the codebooks increases the robustness against diverse image degradations. Nevertheless, these methods require separate training of codebooks for different image categories, which limits their use to specific image categories only (e.g. face, architecture, etc.), and fail to handle arbitrary natural images. In this paper, we propose AdaCode for learning image-adaptive codebooks for class-agnostic image restoration. Instead of learning a single codebook for each image category, we learn a set of basis codebooks. For a given input image, AdaCode learns a weight map with which we compute a weighted combination of these basis codebooks for adaptive image restoration. Intuitively, AdaCode is a more flexible and expressive discrete generative prior than previous work. Experimental results demonstrate that AdaCode achieves state-of-the-art performance on image reconstruction and restoration tasks, including image super-resolution and inpainting.

翻译：近期关于离散生成先验（以码本形式）的研究在图像重建与恢复任务中展现出令人振奋的性能，因为码本所覆盖的离散先验空间增强了对于多样化图像退化的鲁棒性。然而，这些方法需要针对不同图像类别分别训练码本，这限制了它们仅能应用于特定图像类别（如人脸、建筑等），而无法处理任意自然图像。本文提出AdaCode框架，用于学习图像自适应码本以实现类别无关的图像恢复。不同于为每类图像学习单一码本，我们学习一组基础码本。对于给定输入图像，AdaCode学习一个权重图，并通过该权重图计算这些基础码本的加权组合，从而实现自适应图像恢复。直观而言，AdaCode相比先前工作是一种更灵活且更具表达力的离散生成先验。实验结果表明，AdaCode在图像超分辨率与图像修复等图像重建与恢复任务中达到了最先进的性能。