We propose a novel image editing technique that enables 3D manipulations on single images, such as object rotation and translation. Existing 3D-aware image editing approaches typically rely on synthetic multi-view datasets for training specialized models, thus constraining their effectiveness on open-domain images featuring significantly more varied layouts and styles. In contrast, our method directly leverages powerful image diffusion models trained on a broad spectrum of text-image pairs and thus retain their exceptional generalization abilities. This objective is realized through the development of an iterative novel view synthesis and geometry alignment algorithm. The algorithm harnesses diffusion models for dual purposes: they provide appearance prior by predicting novel views of the selected object using estimated depth maps, and they act as a geometry critic by correcting misalignments in 3D shapes across the sampled views. Our method can generate high-quality 3D-aware image edits with large viewpoint transformations and high appearance and shape consistency with the input image, pushing the boundaries of what is possible with single-image 3D-aware editing.

翻译：我们提出一种新颖的图像编辑技术，能够对单张图像实现三维操作，例如物体旋转与平移。现有的三维感知图像编辑方法通常依赖合成多视角数据集训练专用模型，因而限制了其对布局和风格差异显著的开放域图像的有效性。相比之下，我们的方法直接利用在大量文本-图像对上训练的强图像扩散模型，从而保留其卓越的泛化能力。这一目标通过开发一种迭代式新视角合成与几何对齐算法得以实现。该算法利用扩散模型实现双重功能：通过使用估计深度图预测选定物体的新视角来提供外观先验；同时，作为几何批评家，纠正采样视图中三维形状的错位问题。我们的方法能够生成高质量的三维感知图像编辑结果，实现大幅视角变换，并与输入图像保持高度外观与形状一致性，从而拓展了单图像三维感知编辑的可能性边界。