Learning from Demonstration (LfD) is a promising approach to enable Multi-Robot Systems (MRS) to acquire complex skills and behaviors. However, the intricate interactions and coordination challenges in MRS pose significant hurdles for effective LfD. In this paper, we present a novel LfD framework specifically designed for MRS, which leverages visual demonstrations to capture and learn from robot-robot and robot-object interactions. Our framework introduces the concept of Interaction Keypoints (IKs) to transform the visual demonstrations into a representation that facilitates the inference of various skills necessary for the task. The robots then execute the task using sensorimotor actions and reinforcement learning (RL) policies when required. A key feature of our approach is the ability to handle unseen contact-based skills that emerge during the demonstration. In such cases, RL is employed to learn the skill using a classifier-based reward function, eliminating the need for manual reward engineering and ensuring adaptability to environmental changes. We evaluate our framework across a range of mobile robot tasks, covering both behavior-based and contact-based domains. The results demonstrate the effectiveness of our approach in enabling robots to learn complex multi-robot tasks and behaviors from visual demonstrations.

翻译：示范学习（LfD）是实现多机器人系统（MRS）获取复杂技能与行为的一种有前景的方法。然而，MRS中复杂的交互与协调挑战对有效的LfD构成了显著障碍。本文提出了一种专为MRS设计的新型LfD框架，该框架利用视觉示范来捕捉并学习机器人-机器人及机器人-物体之间的交互。我们的框架引入了交互关键点（IKs）概念，将视觉示范转化为便于推断任务所需多种技能的表示形式。随后，机器人通过感觉运动动作及必要时结合强化学习（RL）策略来执行任务。该方法的一个关键特征在于能够处理示范过程中出现的不可预见的接触式技能。在此类情况下，我们采用基于分类器奖励函数的RL进行技能学习，无需人工设计奖励函数，并确保对环境变化的适应性。我们在一系列移动机器人任务中对该框架进行了评估，涵盖了基于行为与基于接触的两类领域。结果表明，该方法能有效使机器人从视觉示范中学习复杂的多机器人任务与行为。