Fleets of robots ingest massive amounts of heterogeneous streaming data silos generated by interacting with their environments, far more than what can be stored or transmitted with ease. At the same time, teams of robots should co-acquire diverse skills through their heterogeneous experiences in varied settings. How can we enable such fleet-level learning without having to transmit or centralize fleet-scale data? In this paper, we investigate policy merging (PoMe) from such distributed heterogeneous datasets as a potential solution. To efficiently merge policies in the fleet setting, we propose FLEET-MERGE, an instantiation of distributed learning that accounts for the permutation invariance that arises when parameterizing the control policies with recurrent neural networks. We show that FLEET-MERGE consolidates the behavior of policies trained on 50 tasks in the Meta-World environment, with good performance on nearly all training tasks at test time. Moreover, we introduce a novel robotic tool-use benchmark, FLEET-TOOLS, for fleet policy learning in compositional and contact-rich robot manipulation tasks, to validate the efficacy of FLEET-MERGE on the benchmark.

翻译：机器人舰队在与环境交互过程中，会产生海量异构数据流，这些数据远超便捷存储或传输的能力。与此同时，机器人团队应通过在不同环境中的异构经验，共同习得多样化技能。如何在不传输或集中舰队规模数据的情况下，实现这种舰队级学习？本文探索了从分布式异构数据集中进行策略合并（PoMe）的潜在解决方案。为有效合并舰队场景下的策略，我们提出FLEET-MERGE——一种分布式学习的实例化方法，其处理了使用循环神经网络参数化控制策略时出现的排列不变性问题。实验表明，FLEET-MERGE能够整合Meta-World环境中50个任务训练所得策略的行为，并在测试时对几乎所有训练任务均表现出良好性能。此外，我们引入了一个新颖的机器人工具使用基准FLEET-TOOLS，用于组合式且富含接触的机器人操作任务中的舰队策略学习，以验证FLEET-MERGE在该基准上的有效性。