Unveiling an empty street from crowded observations captured by in-car cameras is crucial for autonomous driving. However, removing all temporarily static objects, such as stopped vehicles and standing pedestrians, presents a significant challenge. Unlike object-centric 3D inpainting, which relies on thorough observation in a small scene, street scene cases involve long trajectories that differ from previous 3D inpainting tasks. The camera-centric moving environment of captured videos further complicates the task due to the limited degree and time duration of object observation. To address these obstacles, we introduce StreetUnveiler to reconstruct an empty street. StreetUnveiler learns a 3D representation of the empty street from crowded observations. Our representation is based on the hard-label semantic 2D Gaussian Splatting (2DGS) for its scalability and ability to identify Gaussians to be removed. We inpaint rendered image after removing unwanted Gaussians to provide pseudo-labels and subsequently re-optimize the 2DGS. Given its temporal continuous movement, we divide the empty street scene into observed, partial-observed, and unobserved regions, which we propose to locate through a rendered alpha map. This decomposition helps us to minimize the regions that need to be inpainted. To enhance the temporal consistency of the inpainting, we introduce a novel time-reversal framework to inpaint frames in reverse order and use later frames as references for earlier frames to fully utilize the long-trajectory observations. Our experiments conducted on the street scene dataset successfully reconstructed a 3D representation of the empty street. The mesh representation of the empty street can be extracted for further applications. The project page and more visualizations can be found at: https://streetunveiler.github.io

翻译：从车载摄像头拍摄的拥挤观测中揭示空荡街道对于自动驾驶至关重要。然而，移除所有临时静态物体（如停驻车辆和站立行人）是一项重大挑战。与依赖小场景全面观测的以物体为中心的三维修复不同，街道场景涉及长轨迹，这与先前的三维修复任务存在差异。拍摄视频以相机为中心的移动环境，由于物体观测的角度和时间有限，进一步增加了任务的复杂性。为克服这些障碍，我们提出StreetUnveiler来重建空荡街道。StreetUnveiler从拥挤观测中学习空荡街道的三维表示。我们的表示基于硬标签语义二维高斯泼溅（2DGS），因其可扩展性且能识别待移除的高斯元素。我们在移除不需要的高斯元素后对渲染图像进行修复以提供伪标签，随后重新优化2DGS。考虑到其时间连续性运动，我们将空荡街道场景划分为已观测、部分观测和未观测区域，并提出通过渲染的alpha图进行定位。这种分解有助于最小化需要修复的区域。为增强修复的时间一致性，我们引入了一种新颖的时间反转框架，以逆序修复帧，并利用后续帧作为先前帧的参考，从而充分利用长轨迹观测。我们在街道场景数据集上进行的实验成功重建了空荡街道的三维表示。空荡街道的网格表示可被提取以供进一步应用。项目页面及更多可视化内容请访问：https://streetunveiler.github.io