We present Adaptive Skill Coordination (ASC) - an approach for accomplishing long-horizon tasks (e.g., mobile pick-and-place, consisting of navigating to an object, picking it, navigating to another location, placing it, repeating). ASC consists of three components - (1) a library of basic visuomotor skills (navigation, pick, place), (2) a skill coordination policy that chooses which skills are appropriate to use when, and (3) a corrective policy that adapts pre-trained skills when out-of-distribution states are perceived. All components of ASC rely only on onboard visual and proprioceptive sensing, without access to privileged information like pre-built maps or precise object locations, easing real-world deployment. We train ASC in simulated indoor environments, and deploy it zero-shot in two novel real-world environments on the Boston Dynamics Spot robot. ASC achieves near-perfect performance at mobile pick-and-place, succeeding in 59/60 (98%) episodes, while sequentially executing skills succeeds in only 44/60 (73%) episodes. It is robust to hand-off errors, changes in the environment layout, dynamic obstacles (e.g., people), and unexpected disturbances, making it an ideal framework for complex, long-horizon tasks. Supplementary videos available at adaptiveskillcoordination.github.io.

翻译：我们提出自适应技能协调（ASC）——一种用于完成长时域任务（例如移动抓取与放置，包括导航至目标物体、抓取物体、导航至另一位置、放置物体、重复此过程）的方法。ASC包含三个组件：（1）基础视觉运动技能库（导航、抓取、放置），（2）技能协调策略，用于选择在何时使用哪些技能，以及（3）纠错策略，用于在感知到分布外状态时调整预训练技能。ASC的所有组件仅依赖机载视觉与本体感觉，无需访问预先构建的地图或精确物体位置等特权信息，从而简化了实际部署。我们在模拟室内环境中训练ASC，并将其零样本部署在波士顿动力Spot机器人的两个全新真实世界环境中。ASC在移动抓取与放置任务中实现了近乎完美的性能，在60次测试中成功59次（成功率98%），而顺序执行技能的成功率仅为44/60（73%）。该方法对手部交接错误、环境布局变化、动态障碍物（如行人）以及意外干扰具有鲁棒性，使其成为应对复杂长时域任务的理想框架。补充视频请访问 adaptiveskillcoordination.github.io。