This paper introduces MVDiffusion, a simple yet effective method for generating consistent multi-view images from text prompts given pixel-to-pixel correspondences (e.g., perspective crops from a panorama or multi-view images given depth maps and poses). Unlike prior methods that rely on iterative image warping and inpainting, MVDiffusion simultaneously generates all images with a global awareness, effectively addressing the prevalent error accumulation issue. At its core, MVDiffusion processes perspective images in parallel with a pre-trained text-to-image diffusion model, while integrating novel correspondence-aware attention layers to facilitate cross-view interactions. For panorama generation, while only trained with 10k panoramas, MVDiffusion is able to generate high-resolution photorealistic images for arbitrary texts or extrapolate one perspective image to a 360-degree view. For multi-view depth-to-image generation, MVDiffusion demonstrates state-of-the-art performance for texturing a scene mesh. The project page is at https://mvdiffusion.github.io/.

翻译：本文提出MVDiffusion，一种简单而有效的方法，可从给定像素级对应关系（例如，全景图的透视裁剪或基于深度图和姿态的多视图图像）的文本提示生成一致的多视图图像。与依赖迭代图像扭曲和修复的先前方法不同，MVDiffusion以全局意识同时生成所有图像，有效解决了常见的误差累积问题。其核心在于，MVDiffusion利用预训练的文本到图像扩散模型并行处理透视图像，同时集成新颖的对应感知注意力层以促进跨视图交互。在全景图生成方面，尽管仅使用10k张全景图进行训练，MVDiffusion仍能为任意文本生成高分辨率逼真图像，或将一张透视图像外推至360度视野。对于多视图深度到图像生成，MVDiffusion在场景网格纹理化方面展现出最先进的性能。项目页面位于https://mvdiffusion.github.io/。