Learning for Demonstration (LfD) enables robots to acquire new skills by imitating expert demonstrations, allowing users to communicate their instructions in an intuitive manner. Recent progress in LfD often relies on kinesthetic teaching or teleoperation as the medium for users to specify the demonstrations. Kinesthetic teaching requires physical handling of the robot, while teleoperation demands proficiency with additional hardware. This paper introduces an alternative paradigm for LfD called Diagrammatic Teaching. Diagrammatic Teaching aims to teach robots novel skills by prompting the user to sketch out demonstration trajectories on 2D images of the scene, these are then synthesised as a generative model of motion trajectories in 3D task space. Additionally, we present the Ray-tracing Probabilistic Trajectory Learning (RPTL) framework for Diagrammatic Teaching. RPTL extracts time-varying probability densities from the 2D sketches, applies ray-tracing to find corresponding regions in 3D Cartesian space, and fits a probabilistic model of motion trajectories to these regions. New motion trajectories, which mimic those sketched by the user, can then be generated from the probabilistic model. We empirically validate our framework both in simulation and on real robots, which include a fixed-base manipulator and a quadruped-mounted manipulator.

翻译：示范学习（Learning from Demonstration, LfD）使机器人能够通过模仿专家示范来获取新技能，允许用户以直观的方式传达指令。近期LfD的进展通常依赖动觉教学或远程操作作为用户指定示范的媒介。动觉教学需要物理接触机器人，而远程操作则要求用户熟练掌握额外硬件设备。本文提出了名为"图解教学"（Diagrammatic Teaching）的LfD替代范式。图解教学旨在通过引导用户在场景的二维图像上绘制示范轨迹来教授机器人新技能，随后这些轨迹被综合为三维任务空间中运动轨迹的生成模型。此外，我们提出了用于图解教学的射线追踪概率轨迹学习（Ray-tracing Probabilistic Trajectory Learning, RPTL）框架。RPTL从二维草图中提取时变概率密度，应用射线追踪在三维笛卡尔空间中定位对应区域，并为这些区域拟合运动轨迹的概率模型。从该概率模型可生成模仿用户绘制轨迹的新运动轨迹。我们通过仿真实验和真实机器人平台（包括固定基座机械臂和四足机器人搭载机械臂）对该框架进行了实证验证。