Machine learning models are commonly trained end-to-end and in a supervised setting, using paired (input, output) data. Examples include recent super-resolution methods that train on pairs of (low-resolution, high-resolution) images. However, these end-to-end approaches require re-training every time there is a distribution shift in the inputs (e.g., night images vs daylight) or relevant latent variables (e.g., camera blur or hand motion). In this work, we leverage state-of-the-art (SOTA) generative models (here StyleGAN2) for building powerful image priors, which enable application of Bayes' theorem for many downstream reconstruction tasks. Our method, Bayesian Reconstruction through Generative Models (BRGM), uses a single pre-trained generator model to solve different image restoration tasks, i.e., super-resolution and in-painting, by combining it with different forward corruption models. We keep the weights of the generator model fixed, and reconstruct the image by estimating the Bayesian maximum a-posteriori (MAP) estimate over the input latent vector that generated the reconstructed image. We further use variational inference to approximate the posterior distribution over the latent vectors, from which we sample multiple solutions. We demonstrate BRGM on three large and diverse datasets: (i) 60,000 images from the Flick Faces High Quality dataset (ii) 240,000 chest X-rays from MIMIC III and (iii) a combined collection of 5 brain MRI datasets with 7,329 scans. Across all three datasets and without any dataset-specific hyperparameter tuning, our simple approach yields performance competitive with current task-specific state-of-the-art methods on super-resolution and in-painting, while being more generalisable and without requiring any training. Our source code and pre-trained models are available online: https://razvanmarinescu.github.io/brgm/.

翻译：机器学习模型通常经过端对端培训,在监督的环境中使用配对(投入、输出)数据。例子包括最近的超分辨率方法,对相配(低分辨率、高分辨率)图像进行训练。然而,每次投入(例如,夜间图像对日光)或相关潜在变量(例如,照相机模糊或手动)的分布变化时,这些端对端方法都需要再培训。在这项工作中,我们利用最先进的(SteleGAN2) 智能模型(这里是SteleGAN2) 来建立强大的图像前程,这样可以应用Bayes的图像来完成许多下游重建任务。然而,每次输入(低分辨率、高分辨率) 或相关潜在变量(例如,超级分辨率或手动) 或相关潜在变量(例如,相机模糊或手动) 。我们保持发电机模型(Steleglegal-alder) 的重量,并且通过进一步估算Bayesignal-resseral-resserveal 等图像(MA) 估计Bayes' the liveral-liveral exal exal exal exmodal demodal dal dal dal 3,我们用三大数据流数据流流数据流数据流数据流数据流数据流数据流数据流。我们用三个数据流数据流,我们用三个数据流数据流数据流数据流数据。