We present the first application of 3D Gaussian Splatting in monocular SLAM, the most fundamental but the hardest setup for Visual SLAM. Our method, which runs live at 3fps, utilises Gaussians as the only 3D representation, unifying the required representation for accurate, efficient tracking, mapping, and high-quality rendering. Designed for challenging monocular settings, our approach is seamlessly extendable to RGB-D SLAM when an external depth sensor is available. Several innovations are required to continuously reconstruct 3D scenes with high fidelity from a live camera. First, to move beyond the original 3DGS algorithm, which requires accurate poses from an offline Structure from Motion (SfM) system, we formulate camera tracking for 3DGS using direct optimisation against the 3D Gaussians, and show that this enables fast and robust tracking with a wide basin of convergence. Second, by utilising the explicit nature of the Gaussians, we introduce geometric verification and regularisation to handle the ambiguities occurring in incremental 3D dense reconstruction. Finally, we introduce a full SLAM system which not only achieves state-of-the-art results in novel view synthesis and trajectory estimation but also reconstruction of tiny and even transparent objects.

翻译：我们首次将三维高斯溅射应用于单目SLAM，这是视觉SLAM中最基础但最具挑战性的设定。我们的方法以每秒3帧的实时运行速度，将高斯函数作为唯一的三维表示，统一了精确高效追踪、建图及高质量渲染所需的表征形式。该方法专为困难的单目场景设计，当配备外部深度传感器时可无缝扩展至RGB-D SLAM。为实现从实时摄像头持续高保真重建三维场景，我们提出多项创新：首先，突破原始3DGS算法需依赖离线运动恢复结构系统提供精确位姿的限制，我们通过直接优化三维高斯函数来制定相机追踪方案，并证明该方法具有宽收敛域、快速鲁棒的追踪性能；其次，利用高斯的显式特性，引入几何验证与正则化机制以处理增量式三维稠密重建中的歧义性；最终构建完整的SLAM系统，不仅在新型视角合成与轨迹估计方面达到顶尖水平，更可完成微小甚至透明物体的重建。