We propose NEMTO, the first end-to-end neural rendering pipeline to model 3D transparent objects with complex geometry and unknown indices of refraction. Commonly used appearance modeling such as the Disney BSDF model cannot accurately address this challenging problem due to the complex light paths bending through refractions and the strong dependency of surface appearance on illumination. With 2D images of the transparent object as input, our method is capable of high-quality novel view and relighting synthesis. We leverage implicit Signed Distance Functions (SDF) to model the object geometry and propose a refraction-aware ray bending network to model the effects of light refraction within the object. Our ray bending network is more tolerant to geometric inaccuracies than traditional physically-based methods for rendering transparent objects. We provide extensive evaluations on both synthetic and real-world datasets to demonstrate our high-quality synthesis and the applicability of our method.

翻译：我们提出NEMTO，这是首个端到端神经渲染管线，用于建模具有复杂几何形状和未知折射率的3D透明物体。由于光线经折射产生弯曲的复杂路径以及表面外观对照明的高度依赖性，常用的外观建模方法（如Disney BSDF模型）无法精确应对这一挑战性问题。以透明物体的二维图像为输入，我们的方法能够实现高质量的新视角合成和重光照合成。我们利用隐式符号距离函数（SDF）建模物体几何，并提出一种折射感知光线弯曲网络，以模拟光线在物体内部折射产生的效应。与传统的基于物理的透明物体渲染方法相比，我们的光线弯曲网络对几何不精确性具有更高的容忍度。我们在合成数据集和真实世界数据集上进行了广泛评估，以展示该方法的高质量合成效果及其实用性。