Transparent objects are ubiquitous in industry, pharmaceuticals, and households. Grasping and manipulating these objects is a significant challenge for robots. Existing methods have difficulty reconstructing complete depth maps for challenging transparent objects, leaving holes in the depth reconstruction. Recent work has shown neural radiance fields (NeRFs) work well for depth perception in scenes with transparent objects, and these depth maps can be used to grasp transparent objects with high accuracy. NeRF-based depth reconstruction can still struggle with especially challenging transparent objects and lighting conditions. In this work, we propose Residual-NeRF, a method to improve depth perception and training speed for transparent objects. Robots often operate in the same area, such as a kitchen. By first learning a background NeRF of the scene without transparent objects to be manipulated, we reduce the ambiguity faced by learning the changes with the new object. We propose training two additional networks: a residual NeRF learns to infer residual RGB values and densities, and a Mixnet learns how to combine background and residual NeRFs. We contribute synthetic and real experiments that suggest Residual-NeRF improves depth perception of transparent objects. The results on synthetic data suggest Residual-NeRF outperforms the baselines with a 46.1% lower RMSE and a 29.5% lower MAE. Real-world qualitative experiments suggest Residual-NeRF leads to more robust depth maps with less noise and fewer holes. Website: https://residual-nerf.github.io

翻译：透明物体在工业、制药和家庭环境中普遍存在。抓取并操作这些物体对机器人而言是一项重大挑战。现有方法难以重构复杂透明物体的完整深度图，导致深度重建中出现空洞。近期研究表明，神经辐射场（NeRF）在透明物体场景的深度感知中表现优异，生成的深度图可用于高精度抓取透明物体。然而，基于NeRF的深度重建在处理特别复杂的透明物体及光照条件时仍存在困难。本文提出Residual-NeRF方法，旨在提升透明物体的深度感知能力与训练速度。机器人通常在固定区域（如厨房）运行，通过预先生成场景中无待操作透明物体的背景NeRF，可降低学习新物体变化时的歧义性。我们提出训练两个附加网络：残差NeRF学习推断残差RGB值与密度，混合网络学习如何融合背景NeRF与残差NeRF。我们通过合成数据与真实实验证明，Residual-NeRF可提升透明物体的深度感知质量。合成数据结果表明，Residual-NeRF的均方根误差降低46.1%，平均绝对误差降低29.5%，优于基线方法。真实场景定性实验表明，Residual-NeRF可生成噪声更少、空洞更少的鲁棒深度图。网站：https://residual-nerf.github.io