Training a policy that can generalize to unknown objects is a long standing challenge within the field of robotics. The performance of a policy often drops significantly in situations where an object in the scene was not seen during training. To solve this problem, we present NeRF-Aug, a novel method that is capable of teaching a policy to interact with objects that are not present in the dataset. This approach differs from existing approaches by leveraging the speed and photorealism of a neural radiance field for augmentation. NeRF- Aug both creates more photorealistic data and runs 3.83 times faster than existing methods. We demonstrate the effectiveness of our method on 4 tasks with 11 novel objects that have no expert demonstration data. We achieve an average 69.1% success rate increase over existing methods. See video results at https://nerf-aug.github.io.

翻译：训练能够泛化到未知物体的策略是机器人领域长期存在的挑战。当场景中出现训练期间未见过的物体时，策略性能通常会显著下降。为解决此问题，我们提出NeRF-Aug——一种能够教导策略与数据集中未出现物体进行交互的新方法。该方法通过利用神经辐射场的速度与照片级真实感进行数据增强，从而区别于现有方法。NeRF-Aug不仅能生成更具照片真实感的数据，其运行速度更达到现有方法的3.83倍。我们在包含11个无专家演示数据的新颖物体上，通过4项任务验证了本方法的有效性。相较于现有方法，我们实现了平均69.1%的成功率提升。视频结果请参见 https://nerf-aug.github.io。