Differentiable 3D-Gaussian splatting (GS) is emerging as a prominent technique in computer vision and graphics for reconstructing 3D scenes. GS represents a scene as a set of 3D Gaussians with varying opacities and employs a computationally efficient splatting operation along with analytical derivatives to compute the 3D Gaussian parameters given scene images captured from various viewpoints. Unfortunately, capturing surround view ($360^{\circ}$ viewpoint) images is impossible or impractical in many real-world imaging scenarios, including underwater imaging, rooms inside a building, and autonomous navigation. In these restricted baseline imaging scenarios, the GS algorithm suffers from a well-known 'missing cone' problem, which results in poor reconstruction along the depth axis. In this manuscript, we demonstrate that using transient data (from sonars) allows us to address the missing cone problem by sampling high-frequency data along the depth axis. We extend the Gaussian splatting algorithms for two commonly used sonars and propose fusion algorithms that simultaneously utilize RGB camera data and sonar data. Through simulations, emulations, and hardware experiments across various imaging scenarios, we show that the proposed fusion algorithms lead to significantly better novel view synthesis (5 dB improvement in PSNR) and 3D geometry reconstruction (60% lower Chamfer distance).

翻译：可微三维高斯泼溅（GS）正成为计算机视觉与图形学领域用于重建三维场景的重要技术。GS通过一组具有不同不透明度的三维高斯函数表示场景，并采用计算高效的泼溅操作联合解析导数，根据从不同视角捕获的场景图像计算三维高斯参数。然而，在许多实际成像场景中（包括水下成像、建筑内部房间及自主导航等），获取环绕视场（360°视角）图像往往不可行或不切实际。在这些受限基线成像场景中，GS算法面临著名的"缺失锥"问题，导致沿深度轴的重建质量较差。本文证明，利用声纳的瞬态数据可通过沿深度轴采集高频数据来解决缺失锥问题。我们针对两种常用声纳设备扩展了高斯泼溅算法，并提出了同时利用RGB相机数据与声纳数据的融合算法。通过在不同成像场景中进行的仿真、模拟及硬件实验，我们证实所提出的融合算法能显著提升新视角合成质量（PSNR提升5 dB）与三维几何重建精度（Chamfer距离降低60%）。