Loco-manipulation calls for effective whole-body control and contact-rich interactions with the object and the environment. Existing learning-based control frameworks rely on task-specific engineered rewards, training a set of low-level skill policies and explicitly switching between them with a high-level policy or FSM, leading to quasi-static and fragile transitions between skills. In contrast, for solving highly dynamic tasks such as soccer, the robot should run towards the ball, decelerating into an optimal approach configuration to seamlessly switch to dribbling and eventually score a goal - a continuum of smooth motion. To this end, we propose to learn a single Oracle Guided Multi-mode Policy (OGMP) for mastering all the required modes and transition maneuvers to solve uni-object bipedal loco-manipulation tasks. Specifically, we design a multi-mode oracle as a closed loop state-reference generator, viewing it as a hybrid automaton with continuous reference generating dynamics and discrete mode jumps. Given such an oracle, we then train an OGMP through bounded exploration around the generated reference. Furthermore, to enforce the policy to learn the desired sequence of mode transitions, we present a novel task-agnostic mode-switching preference reward that enhances performance. The proposed approach results in successful dynamic loco-manipulation in omnidirectional soccer and box-moving tasks with a 16-DoF bipedal robot HECTOR. Supplementary video results are available at https://www.youtube.com/watch?v=gfDaRqobheg

翻译：移动操作要求有效的全身控制以及与物体和环境的密集接触交互。现有基于学习的控制框架依赖于任务特定的工程化奖励，通过训练一组低层技能策略并由高层策略或有限状态机（FSM）显式切换，导致技能间转换呈现准静态且脆弱。相比之下，为解决如足球等高度动态任务，机器人应跑向球体，减速至最优接近构型以无缝切换至运球并最终射门——构成一个连续平滑的运动流。为此，我们提出学习单一Oracle引导的多模态策略（OGMP），以掌握解决单对象双足移动操作任务所需的所有模态及转换动作。具体而言，我们设计了一个多模态Oracle作为闭环状态参考生成器，将其视为具有连续参考生成动态和离散模态跳变的混合自动机。给定该Oracle，我们通过在生成参考周围进行有界探索来训练OGMP。此外，为强制策略学习期望的模态转换序列，我们提出了一种新颖的任务无关模态切换偏好奖励以提升性能。所提方法在16自由度双足机器人HECTOR上成功实现了全向足球和箱体搬运任务中的动态移动操作。补充视频结果可见于 https://www.youtube.com/watch?v=gfDaRqobheg