In underwater images, most useful features are occluded by water. The extent of the occlusion depends on imaging geometry and can vary even across a sequence of burst images. As a result, 3D reconstruction methods robust on in-air scenes, like Neural Radiance Field methods (NeRFs) or 3D Gaussian Splatting (3DGS), fail on underwater scenes. While a recent underwater adaptation of NeRFs achieved state-of-the-art results, it is impractically slow: reconstruction takes hours and its rendering rate, in frames per second (FPS), is less than 1. Here, we present a new method that takes only a few minutes for reconstruction and renders novel underwater scenes at 140 FPS. Named Gaussian Splashing, our method unifies the strengths and speed of 3DGS with an image formation model for capturing scattering, introducing innovations in the rendering and depth estimation procedures and in the 3DGS loss function. Despite the complexities of underwater adaptation, our method produces images at unparalleled speeds with superior details. Moreover, it reveals distant scene details with far greater clarity than other methods, dramatically improving reconstructed and rendered images. We demonstrate results on existing datasets and a new dataset we have collected. Additional visual results are available at: https://bgu-cs-vil.github.io/gaussiansplashingUW.github.io/ .
翻译:在水下图像中,大部分有效特征被水体所遮挡。遮挡程度取决于成像几何关系,即使在连拍图像序列中也可能存在差异。因此,在陆地场景中表现稳健的三维重建方法(如神经辐射场方法(NeRFs)或三维高斯溅射(3DGS))在水下场景中会失效。尽管近期针对NeRFs的水下适配方法取得了最先进的结果,但其速度极不实用:重建需耗时数小时,且渲染速率(以每秒帧数FPS计)低于1帧。本文提出一种新方法,仅需数分钟即可完成重建,并以140 FPS的速率渲染新视角水下场景。我们将其命名为高斯溅射,该方法融合了3DGS的优势与速度,并结合捕获散射效应的成像模型,在渲染与深度估计流程以及3DGS损失函数中引入了创新性改进。尽管水下适配具有复杂性,本方法仍能以无与伦比的速度生成细节更丰富的图像。此外,相较于其他方法,本方法能以更清晰的视觉效果呈现远处场景细节,显著提升重建与渲染图像质量。我们在现有数据集及新采集的数据集上验证了实验结果。更多可视化结果请访问:https://bgu-cs-vil.github.io/gaussiansplashingUW.github.io/。