We present Adaptive Skill Coordination (ASC) -- an approach for accomplishing long-horizon tasks like mobile pick-and-place (i.e., navigating to an object, picking it, navigating to another location, and placing it). ASC consists of three components -- (1) a library of basic visuomotor skills (navigation, pick, place), (2) a skill coordination policy that chooses which skill to use when, and (3) a corrective policy that adapts pre-trained skills in out-of-distribution states. All components of ASC rely only on onboard visual and proprioceptive sensing, without requiring detailed maps with obstacle layouts or precise object locations, easing real-world deployment. We train ASC in simulated indoor environments, and deploy it zero-shot (without any real-world experience or fine-tuning) on the Boston Dynamics Spot robot in eight novel real-world environments (one apartment, one lab, two microkitchens, two lounges, one office space, one outdoor courtyard). In rigorous quantitative comparisons in two environments, ASC achieves near-perfect performance (59/60 episodes, or 98%), while sequentially executing skills succeeds in only 44/60 (73%) episodes. Extensive perturbation experiments show that ASC is robust to hand-off errors, changes in the environment layout, dynamic obstacles (e.g., people), and unexpected disturbances. Supplementary videos at adaptiveskillcoordination.github.io.

翻译：我们提出自适应技能协调（ASC）——一种完成移动拾取-放置等长时域任务（即导航至物体、拾取物体、导航至另一位置、放置物体）的方法。ASC包含三个组件：(1) 基础视觉运动技能库（导航、拾取、放置）；(2) 技能协调策略，用于选择何时使用何种技能；(3) 校正策略，用于在分布外状态下调整预训练技能。所有ASC组件仅依赖板载视觉和本体感知，无需包含障碍物布局的详细地图或精确物体位置，从而简化了实际部署。我们在模拟室内环境中训练ASC，并零样本（无任何真实世界经验或微调）部署于波士顿动力Spot机器人上，在八个新真实世界环境（一处公寓、一间实验室、两个微型厨房、两个休息室、一个办公空间、一个室外庭院）中进行测试。在两个环境中的严格定量对比中，ASC实现了近乎完美的性能（59/60个回合，即98%），而顺序执行技能仅在44/60个回合（73%）中成功。大量扰动实验表明，ASC对交接错误、环境布局变化、动态障碍物（如人员）以及意外干扰具有鲁棒性。补充视频见adaptiveskillcoordination.github.io。