Differentiable 3D-Gaussian splatting (GS) is emerging as a prominent technique in computer vision and graphics for reconstructing 3D scenes. GS represents a scene as a set of 3D Gaussians with varying opacities and employs a computationally efficient splatting operation along with analytical derivatives to compute the 3D Gaussian parameters given scene images captured from various viewpoints. Unfortunately, capturing surround view ($360^{\circ}$ viewpoint) images is impossible or impractical in many real-world imaging scenarios, including underwater imaging, rooms inside a building, and autonomous navigation. In these restricted baseline imaging scenarios, the GS algorithm suffers from a well-known 'missing cone' problem, which results in poor reconstruction along the depth axis. In this manuscript, we demonstrate that using transient data (from sonars) allows us to address the missing cone problem by sampling high-frequency data along the depth axis. We extend the Gaussian splatting algorithms for two commonly used sonars and propose fusion algorithms that simultaneously utilize RGB camera data and sonar data. Through simulations, emulations, and hardware experiments across various imaging scenarios, we show that the proposed fusion algorithms lead to significantly better novel view synthesis (5 dB improvement in PSNR) and 3D geometry reconstruction (60% lower Chamfer distance).

翻译：可微分三维高斯溅射（GS）正逐渐成为计算机视觉与图形学中用于三维场景重建的重要技术。GS将场景表示为一组具有不同不透明度的三维高斯分布，并采用计算高效的溅射操作及解析导数，根据从不同视角采集的场景图像计算三维高斯参数。然而，在许多实际成像场景中（包括水下成像、建筑内部空间及自主导航），采集环绕视角（360°视角）图像是不可能的或不切实际的。在这些受限基线成像场景中，GS算法存在众所周知的“缺失锥”问题，导致沿深度轴的重建效果较差。本文证明，使用瞬态数据（来自声纳）能够通过沿深度轴采样高频数据来解决缺失锥问题。我们针对两种常用声纳扩展了高斯溅射算法，并提出了同时利用RGB相机数据与声纳数据的融合算法。通过在不同成像场景中进行仿真、模拟和硬件实验，我们证明所提出的融合算法能显著提升新视角合成质量（PSNR提高5 dB）和三维几何重建精度（倒角距离降低60%）。