In collaborative human-robot manipulation, a robot must predict human intents and adapt its actions accordingly to smoothly execute tasks. However, the human's intent in turn depends on actions the robot takes, creating a chicken-or-egg problem. Prior methods ignore such inter-dependency and instead train marginal intent prediction models independent of robot actions. This is because training conditional models is hard given a lack of paired human-robot interaction datasets. Can we instead leverage large-scale human-human interaction data that is more easily accessible? Our key insight is to exploit a correspondence between human and robot actions that enables transfer learning from human-human to human-robot data. We propose a novel architecture, InteRACT, that pre-trains a conditional intent prediction model on large human-human datasets and fine-tunes on a small human-robot dataset. We evaluate on a set of real-world collaborative human-robot manipulation tasks and show that our conditional model improves over various marginal baselines. We also introduce new techniques to tele-operate a 7-DoF robot arm and collect a diverse range of human-robot collaborative manipulation data, which we open-source.

翻译：在人机协作操作中，机器人必须预测人类意图并相应地调整自身动作，以顺利执行任务。然而，人类的意图又取决于机器人采取的动作，这形成了一个“鸡生蛋还是蛋生鸡”的难题。现有方法忽略了这种相互依赖性，转而训练独立于机器人动作的边缘意图预测模型。这是因为缺乏配对的人机交互数据集，导致训练条件模型十分困难。我们能否转而利用更易获取的大规模人-人交互数据？我们的核心见解在于利用人类动作与机器人动作之间的对应关系，实现从人-人数据到人-机器人数据的迁移学习。我们提出了一种新颖的架构InteRACT，该架构首先在大型人-人数据集上预训练条件意图预测模型，然后在小型人-机器人数据集上进行微调。我们在真实世界人机协作操作任务集上进行了评估，结果表明我们的条件模型优于多种边缘基线方法。我们还引入了新技术来遥操作一个7自由度机械臂，并收集了多样化的人机协作操作数据，相关数据已开源发布。