Robot co-design, where the morphology of a robot is optimized jointly with a learned policy to solve a specific task, is an emerging area of research. It holds particular promise for soft robots, which are amenable to novel manufacturing techniques that can realize learned morphologies and actuators. Inspired by nature and recent novel robot designs, we propose to go a step further and explore the novel reconfigurable robots, defined as robots that can change their morphology within their lifetime. We formalize control of reconfigurable soft robots as a high-dimensional reinforcement learning (RL) problem. We unify morphology change, locomotion, and environment interaction in the same action space, and introduce an appropriate, coarse-to-fine curriculum that enables us to discover policies that accomplish fine-grained control of the resulting robots. We also introduce DittoGym, a comprehensive RL benchmark for reconfigurable soft robots that require fine-grained morphology changes to accomplish the tasks. Finally, we evaluate our proposed coarse-to-fine algorithm on DittoGym and demonstrate robots that learn to change their morphology several times within a sequence, uniquely enabled by our RL algorithm. More results are available at https://dittogym.github.io.

翻译：机器人协同设计——将机器人的形态与学习策略联合优化以解决特定任务——是一个新兴的研究领域。该方法对软体机器人尤为适用，因其可借助新型制造技术实现学习到的形态与执行器。受自然界及近期新型机器人设计的启发，我们提议更进一步，探索可重构机器人，即在其生命周期内能改变自身形态的机器人。我们将可重构软体机器人的控制形式化为一个高维强化学习问题，将形态变化、运动和环境交互统一到同一动作空间中，并引入一种恰当的由粗到精的训练课程，使我们能够发现实现对所得机器人精细控制的策略。我们还介绍了DittoGym——一个面向可重构软体机器人的综合性强化学习基准，这类机器人需要精细的形态变化以完成任务。最后，我们在DittoGym上评估了所提出的由粗到精算法，并演示了机器人学会在序列中多次改变形态，这完全得益于我们的强化学习算法。更多结果详见https://dittogym.github.io。