It has been a long-standing research goal to endow robot hands with human-level dexterity. Bi-manual robot piano playing constitutes a task that combines challenges from dynamic tasks, such as generating fast while precise motions, with slower but contact-rich manipulation problems. Although reinforcement learning based approaches have shown promising results in single-task performance, these methods struggle in a multi-song setting. Our work aims to close this gap and, thereby, enable imitation learning approaches for robot piano playing at scale. To this end, we introduce the Robot Piano 1 Million (RP1M) dataset, containing bi-manual robot piano playing motion data of more than one million trajectories. We formulate finger placements as an optimal transport problem, thus, enabling automatic annotation of vast amounts of unlabeled songs. Benchmarking existing imitation learning approaches shows that such approaches reach state-of-the-art robot piano playing performance by leveraging RP1M.

翻译：赋予机器人手以人类水平的灵巧性一直是一个长期的研究目标。双手机器人弹钢琴这一任务，结合了动态任务（如生成快速而精确的运动）与速度较慢但接触丰富的操作问题的挑战。尽管基于强化学习的方法在单一任务性能上已显示出有希望的结果，但这些方法在多首曲目的场景中表现不佳。我们的工作旨在弥合这一差距，从而使得模仿学习方法能够大规模应用于机器人弹钢琴。为此，我们引入了机器人钢琴百万（RP1M）数据集，其中包含超过一百万条轨迹的双手机器人弹钢琴运动数据。我们将手指放置问题表述为一个最优传输问题，从而实现了对大量未标记曲目的自动标注。对现有模仿学习方法的基准测试表明，通过利用RP1M数据集，此类方法达到了最先进的机器人弹钢琴性能。