Developing the next generation of household robot helpers requires combining locomotion and interaction capabilities, which is generally referred to as mobile manipulation (MoMa). MoMa tasks are difficult due to the large action space of the robot and the common multi-objective nature of the task, e.g., efficiently reaching a goal while avoiding obstacles. Current approaches often segregate tasks into navigation without manipulation and stationary manipulation without locomotion by manually matching parts of the action space to MoMa sub-objectives (e.g. base actions for locomotion objectives and arm actions for manipulation). This solution prevents simultaneous combinations of locomotion and interaction degrees of freedom and requires human domain knowledge for both partitioning the action space and matching the action parts to the sub-objectives. In this paper, we introduce Causal MoMa, a new framework to train policies for typical MoMa tasks that makes use of the most favorable subspace of the robot's action space to address each sub-objective. Causal MoMa automatically discovers the causal dependencies between actions and terms of the reward function and exploits these dependencies in a causal policy learning procedure that reduces gradient variance compared to previous state-of-the-art policy gradient algorithms, improving convergence and results. We evaluate the performance of Causal MoMa on three types of simulated robots across different MoMa tasks and demonstrate success in transferring the policies trained in simulation directly to a real robot, where our agent is able to follow moving goals and react to dynamic obstacles while simultaneously and synergistically controlling the whole-body: base, arm, and head. More information at https://sites.google.com/view/causal-moma.

翻译：开发下一代家庭机器人助手需要结合移动与交互能力，这通常被称为移动操作（MoMa）。由于机器人动作空间庞大且任务通常具有多目标特性（例如在高效到达目标的同时避开障碍物），MoMa任务具有较大难度。现有方法通常通过手动将动作空间部分与MoMa子目标匹配（例如，基础动作对应移动目标，手臂动作对应操作目标），将任务划分为不含操作的导航和不含移动的静态操作。这种解决方案阻碍了移动与操作自由度的同步组合，同时需要人类领域知识来完成动作空间划分及动作与子目标的匹配。本文提出Causal MoMa——一种针对典型MoMa任务的新策略训练框架，该框架利用机器人动作空间中最有利的子空间来处理每个子目标。Causal MoMa能自动发现动作与奖励函数项之间的因果依赖关系，并在因果策略学习过程中利用这些依赖关系，相比现有最先进的策略梯度算法减少了梯度方差，从而提升收敛速度与结果质量。我们在三种仿真机器人上针对不同MoMa任务评估了Causal MoMa的性能，并成功将仿真训练策略直接迁移至真实机器人。在真实场景中，我们的智能体能够同时协同控制全身（底盘、机械臂和头部）追踪移动目标并应对动态障碍物。更多信息参见https://sites.google.com/view/causal-moma。