We present Adaptive Skill Coordination (ASC) - an approach for accomplishing long-horizon tasks (e.g., mobile pick-and-place, consisting of navigating to an object, picking it, navigating to another location, placing it, repeating). ASC consists of three components - (1) a library of basic visuomotor skills (navigation, pick, place), (2) a skill coordination policy that chooses which skills are appropriate to use when, and (3) a corrective policy that adapts pre-trained skills when out-of-distribution states are perceived. All components of ASC rely only on onboard visual and proprioceptive sensing, without access to privileged information like pre-built maps or precise object locations, easing real-world deployment. We train ASC in simulated indoor environments, and deploy it zero-shot in two novel real-world environments on the Boston Dynamics Spot robot. ASC achieves near-perfect performance at mobile pick-and-place, succeeding in 59/60 (98%) episodes, while sequentially executing skills succeeds in only 44/60 (73%) episodes. It is robust to hand-off errors, changes in the environment layout, dynamic obstacles (e.g., people), and unexpected disturbances, making it an ideal framework for complex, long-horizon tasks. Supplementary videos available at adaptiveskillcoordination.github.io.

翻译：我们提出了一种 Adaptive Skill Coordination (ASC) 方法，用于完成长时程任务（例如，移动拾取和放置任务，包括导航到一个物体、拾取它、导航到另一个位置、放置它、并重复）。ASC 由三个组成部分组成：（1）基本视觉动作技能库（导航、拾取、放置），（2）技能协调策略，在何时选择适当的技能使用，（3）修正策略，在感知到超出分布状态时适应预先训练的技能。 ASC 的所有组件仅依靠机载视觉和本体感知，没有访问特权信息，例如预构建的地图或精确的物体位置，使其易于在真实环境中部署。我们在模拟室内环境中训练 ASC，并在 Boston Dynamics Spot 机器人上的两个新型真实环境中零样本部署。ASC在移动拾取和放置方面实现了近乎完美的性能，在59/60（98%）个episode中成功，而顺序执行技能则只在44/60（73%）个episode中成功。它对于交接错误、环境布局的变化、动态障碍物（例如人员）和意外干扰具有很强的鲁棒性，使其成为处理复杂、长时程任务的理想框架。补充视频可在 adaptiveskillcoordination.github.io 查看。