Learning for Demonstration (LfD) enables robots to acquire new skills by imitating expert demonstrations, allowing users to communicate their instructions in an intuitive manner. Recent progress in LfD often relies on kinesthetic teaching or teleoperation as the medium for users to specify the demonstrations. Kinesthetic teaching requires physical handling of the robot, while teleoperation demands proficiency with additional hardware. This paper introduces an alternative paradigm for LfD called Diagrammatic Teaching. Diagrammatic Teaching aims to teach robots novel skills by prompting the user to sketch out demonstration trajectories on 2D images of the scene, these are then synthesised as a generative model of motion trajectories in 3D task space. Additionally, we present the Ray-tracing Probabilistic Trajectory Learning (RPTL) framework for Diagrammatic Teaching. RPTL extracts time-varying probability densities from the 2D sketches, applies ray-tracing to find corresponding regions in 3D Cartesian space, and fits a probabilistic model of motion trajectories to these regions. New motion trajectories, which mimic those sketched by the user, can then be generated from the probabilistic model. We empirically validate our framework both in simulation and on real robots, which include a fixed-base manipulator and a quadruped-mounted manipulator.

翻译：示教学习（LfD）使机器人能够通过模仿专家示范来获取新技能，允许用户以直观方式传达指令。当前LfD的研究进展常依赖动觉示教或远程操作作为用户指定示范的媒介。动觉示教需要物理接触机器人，而远程操作则要求用户熟练使用额外硬件。本文提出了一种名为图解教学的新型LfD范式。图解教学旨在通过引导用户在场景二维图像上绘制示范轨迹来教授机器人新技能，这些轨迹随后被综合为三维任务空间中运动轨迹的生成模型。此外，我们针对图解教学提出了射线追踪概率轨迹学习（RPTL）框架。RPTL从二维草图中提取时变概率密度，应用射线追踪确定三维笛卡尔空间中的对应区域，并拟合这些区域中运动轨迹的概率模型。从该概率模型可生成模仿用户草图的新运动轨迹。我们通过仿真实验和真实机器人（包括固定基座机械臂和四足机器人搭载机械臂）的经验验证了该框架的有效性。