This paper presents a novel Learning from Demonstration (LfD) method that uses neural fields to learn new skills efficiently and accurately. It achieves this by utilizing a shared embedding to learn both scene and motion representations in a generative way. Our method smoothly maps each expert demonstration to a scene-motion embedding and learns to model them without requiring hand-crafted task parameters or large datasets. It achieves data efficiency by enforcing scene and motion generation to be smooth with respect to changes in the embedding space. At inference time, our method can retrieve scene-motion embeddings using test time optimization, and generate precise motion trajectories for novel scenes. The proposed method is versatile and can employ images, 3D shapes, and any other scene representations that can be modeled using neural fields. Additionally, it can generate both end-effector positions and joint angle-based trajectories. Our method is evaluated on tasks that require accurate motion trajectory generation, where the underlying task parametrization is based on object positions and geometric scene changes. Experimental results demonstrate that the proposed method outperforms the baseline approaches and generalizes to novel scenes. Furthermore, in real-world experiments, we show that our method can successfully model multi-valued trajectories, it is robust to the distractor objects introduced at inference time, and it can generate 6D motions.

翻译：本文提出一种新颖的示教学习（LfD）方法，通过神经场高效且准确地学习新技能。该方法利用共享嵌入以生成方式同时学习场景与运动的表征。我们提出的方法将每个专家示教平滑映射至场景-运动嵌入，并无需人工设计的任务参数或大规模数据集即可建模。通过强制场景与运动生成在嵌入空间变化中保持平滑，实现数据高效性。在推理阶段，该方法可通过测试时优化检索场景-运动嵌入，并为新场景生成精确运动轨迹。所提方法具有通用性，可处理图像、三维形状及任何可用神经场建模的场景表征，并能同时生成末端执行器位置与关节角轨迹。我们在需要精确运动轨迹生成的任务上评估该方法，其任务参数化基于物体位置与几何场景变化。实验结果表明，该方法在性能上优于基线方法，并能泛化至新场景。此外，实际实验证明该方法可成功建模多值轨迹，对推理时引入的干扰物体具有鲁棒性，并能生成六维运动。