We present a novel method, Aerial Diffusion, for generating aerial views from a single ground-view image using text guidance. Aerial Diffusion leverages a pretrained text-image diffusion model for prior knowledge. We address two main challenges corresponding to domain gap between the ground-view and the aerial view and the two views being far apart in the text-image embedding manifold. Our approach uses a homography inspired by inverse perspective mapping prior to finetuning the pretrained diffusion model. Additionally, using the text corresponding to the ground-view to finetune the model helps us capture the details in the ground-view image at a relatively low bias towards the ground-view image. Aerial Diffusion uses an alternating sampling strategy to compute the optimal solution on complex high-dimensional manifold and generate a high-fidelity (w.r.t. ground view) aerial image. We demonstrate the quality and versatility of Aerial Diffusion on a plethora of images from various domains including nature, human actions, indoor scenes, etc. We qualitatively prove the effectiveness of our method with extensive ablations and comparisons. To the best of our knowledge, Aerial Diffusion is the first approach that performs ground-to-aerial translation in an unsupervised manner.

翻译：我们提出了一种名为Aerial Diffusion的新方法，用于通过文本引导从单张地面视角图像生成航拍视角图像。该方法利用预训练的文本-图像扩散模型获取先验知识，并针对性地解决了两个主要挑战：地面视角与航拍视角之间的领域差异，以及两种视角在文本-图像嵌入流形上的距离过大问题。我们的方法在微调预训练扩散模型前，采用基于逆透视映射的单应变换技术。此外，使用与地面视角对应的文本进行模型微调，有助于在保持较低地面视角图像偏差的同时捕获图像细节。Aerial Diffusion通过交替采样策略在复杂高维流形上求解最优解，生成高保真度（相对于地面视角）的航拍图像。我们通过涵盖自然场景、人类行为、室内场景等多领域的大量图像展示了该方法的质量与多用途性。通过广泛的消融实验与对比，定性地验证了方法的有效性。据我们所知，Aerial Diffusion是首个以无监督方式实现地面到航拍视角转换的方法。