Volumetric phenomena, such as clouds and fog, present a significant challenge for 3D reconstruction systems due to their translucent nature and their complex interactions with light. Conventional techniques for reconstructing scattering volumes rely on controlled setups, limiting practical applications. This paper introduces an approach to reconstructing volumes from a few input stereo pairs. We propose a novel deep learning framework that integrates a deep stereo model with a 3D Convolutional Neural Network (3D CNN) and an advection module, capable of capturing the shape and dynamics of volumes. The stereo depths are used to carve empty space around volumes, providing the 3D CNN with a prior for coping with the lack of input views. Refining our output, the advection module leverages the temporal evolution of the medium, providing a mechanism to infer motion and improve temporal consistency. The efficacy of our system is demonstrated through its ability to estimate density and velocity fields of large-scale volumes, in this case, clouds, from a sparse set of stereo image pairs.

翻译：体现象（如云和雾）由于其半透明性质以及与光的复杂相互作用，给三维重建系统带来了重大挑战。传统的散射体重建技术依赖于受控实验设置，限制了实际应用。本文提出了一种利用少量输入立体像对重建体的方法。我们设计了一个新颖的深度学习框架，该框架将深度立体模型、三维卷积神经网络（3D CNN）与平流模块相结合，能够捕捉体的形状和动态。立体深度被用于雕刻体周围的空间，为3D CNN提供先验信息以应对输入视角数量不足的问题。通过优化输出，平流模块利用介质的时空演化，提供了一种推断运动并改善时间一致性的机制。我们的系统效能通过其从稀疏立体像对中估计大规模体（此处为云）密度场和速度场的能力得到验证。