Inspired by the necessity of morphological adaptation in animals, a growing body of work has attempted to expand robot training to encompass physical aspects of a robot's design. However, reinforcement learning methods capable of optimizing the 3D morphology of a robot have been restricted to reorienting or resizing the limbs of a predetermined and static topological genus. Here we show policy gradients for designing freeform robots with arbitrary external and internal structure. This is achieved through actions that deposit or remove bundles of atomic building blocks to form higher-level nonparametric macrostructures such as appendages, organs and cavities. Although results are provided for open loop control only, we discuss how this method could be adapted for closed loop control and sim2real transfer to physical machines in future.

翻译：受动物形态适应必要性的启发，越来越多的研究尝试将机器人训练扩展到涵盖机器人设计的物理方面。然而，能够优化机器人三维形态的强化学习方法，一直局限于在预设且静态的拓扑类型中重新定向或调整肢体尺寸。本文展示了用于设计具有任意外部与内部结构的自由形态机器人的策略梯度方法。这是通过采用沉积或移除原子级构建块束的动作来实现的，这些动作可形成更高层次的非参数化宏观结构，例如附属肢体、器官和腔体。尽管本文仅提供开环控制的结果，我们讨论了该方法未来如何适用于闭环控制以及向物理机器的仿真到现实迁移。