We study the choice of action space in robot manipulation learning and sim-to-real transfer. We define metrics that assess the performance, and examine the emerging properties in the different action spaces. We train over 250 reinforcement learning~(RL) agents in simulated reaching and pushing tasks, using 13 different control spaces. The choice of spaces spans combinations of common action space design characteristics. We evaluate the training performance in simulation and the transfer to a real-world environment. We identify good and bad characteristics of robotic action spaces and make recommendations for future designs. Our findings have important implications for the design of RL algorithms for robot manipulation tasks, and highlight the need for careful consideration of action spaces when training and transferring RL agents for real-world robotics.

翻译：我们研究了机器人操作学习及仿真到现实迁移中动作空间的选择问题。我们定义了评估性能的度量标准，并考察了不同动作空间中出现的新特性。我们使用13种不同的控制空间，在模拟的到达和推动任务中训练了超过250个强化学习智能体。这些动作空间的选择涵盖了常见动作空间设计特征的组合。我们评估了在模拟环境中的训练性能以及向真实世界环境迁移的效果。我们识别出机器人动作空间的优劣特征，并为未来设计提出了建议。我们的研究结果对机器人操作任务中强化学习算法的设计具有重要启示，并强调了在训练和迁移用于真实世界机器人的强化学习智能体时，需审慎考量动作空间。