This paper studies the task of SatStreet-view synthesis, which aims to render photorealistic street-view panorama images and videos given any satellite image and specified camera positions or trajectories. We formulate to learn neural radiance field from paired images captured from satellite and street viewpoints, which comes to be a challenging learning problem due to the sparse-view natural and the extremely-large viewpoint changes between satellite and street-view images. We tackle the challenges based on a task-specific observation that street-view specific elements, including the sky and illumination effects are only visible in street-view panoramas, and present a novel approach Sat2Density++ to accomplish the goal of photo-realistic street-view panoramas rendering by modeling these street-view specific in neural networks. In the experiments, our method is testified on both urban and suburban scene datasets, demonstrating that Sat2Density++ is capable of rendering photorealistic street-view panoramas that are consistent across multiple views and faithful to the satellite image.

翻译：本文研究SatStreet-view合成任务，其目标是在给定任意卫星图像及指定相机位置或轨迹的条件下，渲染具有照片级真实感的街景全景图像与视频。我们提出从卫星与街景视角的成对图像中学习神经辐射场，由于稀疏视图特性以及卫星与街景图像间极端的视角变化，这成为一个具有挑战性的学习问题。我们基于任务特定的观察来应对这些挑战：街景特有要素（包括天空与光照效果）仅在全景街景中可见，并提出一种新颖方法Sat2Density++，通过神经网络建模这些街景特定要素来实现照片级真实街景全景的渲染。实验中，我们的方法在城市与郊区场景数据集上均得到验证，结果表明Sat2Density++能够渲染出多视角一致且忠实于卫星图像的照片级真实街景全景。