Underwater perception and 3D surface reconstruction are challenging problems with broad applications in construction, security, marine archaeology, and environmental monitoring. Treacherous operating conditions, fragile surroundings, and limited navigation control often dictate that submersibles restrict their range of motion and, thus, the baseline over which they can capture measurements. In the context of 3D scene reconstruction, it is well-known that smaller baselines make reconstruction more challenging. Our work develops a physics-based multimodal acoustic-optical neural surface reconstruction framework (AONeuS) capable of effectively integrating high-resolution RGB measurements with low-resolution depth-resolved imaging sonar measurements. By fusing these complementary modalities, our framework can reconstruct accurate high-resolution 3D surfaces from measurements captured over heavily-restricted baselines. Through extensive simulations and in-lab experiments, we demonstrate that AONeuS dramatically outperforms recent RGB-only and sonar-only inverse-differentiable-rendering--based surface reconstruction methods. A website visualizing the results of our paper is located at this address: https://aoneus.github.io/

翻译：水下感知与三维表面重建是具有广泛应用前景的挑战性问题，涵盖建筑、安防、海洋考古及环境监测等领域。恶劣作业条件、脆弱周边环境及有限的导航控制能力，常迫使水下航行器限制其运动范围，进而缩减可采集数据的基线长度。在三维场景重建领域，已知较小的基线会显著增加重建难度。本研究提出了一种基于物理机制的声光多模态神经表面重建框架（AONeuS），能够有效融合高分辨率RGB测量值与低分辨率深度解析成像声呐数据。通过整合这两种互补模态，本框架可从严格受限基线条件下采集的测量数据中重建出高精度三维表面。大量仿真与实验室实验表明，AONeuS在性能上显著超越近期基于RGB或声呐的逆可微渲染表面重建方法。论文成果可视化网站详见：https://aoneus.github.io/