Learning for Demonstration (LfD) enables robots to acquire new skills by imitating expert demonstrations, allowing users to communicate their instructions in an intuitive manner. Recent progress in LfD often relies on kinesthetic teaching or teleoperation as the medium for users to specify the demonstrations. Kinesthetic teaching requires physical handling of the robot, while teleoperation demands proficiency with additional hardware. This paper introduces an alternative paradigm for LfD called Diagrammatic Teaching. Diagrammatic Teaching aims to teach robots novel skills by prompting the user to sketch out demonstration trajectories on 2D images of the scene, these are then synthesised as a generative model of motion trajectories in 3D task space. Additionally, we present the Ray-tracing Probabilistic Trajectory Learning (RPTL) framework for Diagrammatic Teaching. RPTL extracts time-varying probability densities from the 2D sketches, applies ray-tracing to find corresponding regions in 3D Cartesian space, and fits a probabilistic model of motion trajectories to these regions. New motion trajectories, which mimic those sketched by the user, can then be generated from the probabilistic model. We empirically validate our framework both in simulation and on real robots, which include a fixed-base manipulator and a quadruped-mounted manipulator.

翻译：示教学习（LfD）使机器人能够通过模仿专家示范获取新技能，从而允许用户以直观方式传达指令。近期LfD的进展通常依赖动觉教学或遥操作作为用户指定示范的中介方式。动觉教学需要物理操作机器人，而遥操作则要求用户熟练掌握附加硬件。本文提出一种称为"图解教学"的LfD替代范式。图解教学旨在通过提示用户在场景二维图像上绘制示范轨迹来教授机器人新技能，这些轨迹随后被合成为三维任务空间中运动轨迹的生成模型。此外，我们针对图解教学提出了射线追踪概率轨迹学习（RPTL）框架。RPTL从二维草图中提取时变概率密度，应用射线追踪在三维笛卡尔空间中定位对应区域，并拟合这些区域的运动轨迹概率模型。该概率模型可生成模仿用户草图的新运动轨迹。我们通过仿真实验和真实机器人（包括固定基座机械臂与四足平台机械臂）验证了该框架的有效性。