Despite the promising results of multi-view reconstruction, the recent neural rendering-based methods, such as implicit surface rendering (IDR) and volume rendering (NeuS), not only incur a heavy computational burden on training but also have the difficulties in disentangling the geometric and appearance. Although having achieved faster training speed than implicit representation and hash coding, the explicit voxel-based method obtains the inferior results on recovering surface. To address these challenges, we propose an effective mesh-based neural rendering approach, named FastMESH, which only samples at the intersection of ray and mesh. A coarse-to-fine scheme is introduced to efficiently extract the initial mesh by space carving. More importantly, we suggest a hexagonal mesh model to preserve surface regularity by constraining the second-order derivatives of vertices, where only low level of positional encoding is engaged for neural rendering. The experiments demonstrate that our approach achieves the state-of-the-art results on both reconstruction and novel view synthesis. Besides, we obtain 10-fold acceleration on training comparing to the implicit representation-based methods.
翻译:尽管多视图重建取得了令人鼓舞的结果,但近年基于神经渲染的方法(如隐式表面渲染IDR和体素渲染NeuS)不仅在训练中带来沉重的计算负担,而且难以分离几何与外观信息。尽管显式体素方法在训练速度上优于隐式表示和哈希编码,但在表面恢复方面性能较差。为解决这些问题,我们提出一种高效的基于网格的神经渲染方法——FastMESH,该方法仅在光线与网格的交点处采样。我们引入一种从粗到细的方案,通过空间雕刻高效提取初始网格。更重要的是,我们提出了一种六边形网格模型,通过约束顶点的二阶导数保持表面规则性,该模型在神经渲染中仅需低水平的位置编码。实验表明,我们的方法在重建和新视角合成方面均达到了最先进水平。此外,与基于隐式表示的方法相比,我们的训练速度提升了10倍。