Solving long sequential tasks poses a significant challenge in embodied artificial intelligence. Enabling a robotic system to perform diverse sequential tasks with a broad range of manipulation skills is an active area of research. In this work, we present a Hybrid Hierarchical Learning framework, the Robotic Manipulation Network (ROMAN), to address the challenge of solving multiple complex tasks over long time horizons in robotic manipulation. ROMAN achieves task versatility and robust failure recovery by integrating behavioural cloning, imitation learning, and reinforcement learning. It consists of a central manipulation network that coordinates an ensemble of various neural networks, each specialising in distinct re-combinable sub-tasks to generate their correct in-sequence actions for solving complex long-horizon manipulation tasks. Experimental results show that by orchestrating and activating these specialised manipulation experts, ROMAN generates correct sequential activations for accomplishing long sequences of sophisticated manipulation tasks and achieving adaptive behaviours beyond demonstrations, while exhibiting robustness to various sensory noises. These results demonstrate the significance and versatility of ROMAN's dynamic adaptability featuring autonomous failure recovery capabilities, and highlight its potential for various autonomous manipulation tasks that demand adaptive motor skills.

翻译：解决长序列任务对具身人工智能提出了重大挑战。使机器人系统能够通过广泛的操作技能执行多样化的序列任务，是当前研究活跃的领域。本文提出了一种混合分层学习框架——机器人操作网络（ROMAN），以应对机器人操作中长时间跨度内多个复杂任务的求解挑战。ROMAN通过融合行为克隆、模仿学习与强化学习，实现了任务灵活性与鲁棒的故障恢复能力。该框架包含一个中央操作网络，用于协调一组专门处理不同可重组子任务的多样化神经网络，使其能够针对复杂长时操作任务，按序生成正确的子任务动作序列。实验结果表明，通过编排与激活这些专门化的操作专家模块，ROMAN能够针对复杂操作任务的长序列生成正确的序贯激活模式，并在超越示范行为的同时展现出对各类感知噪声的鲁棒性。这些结果证实了ROMAN动态适应性（具备自主故障恢复能力）的重要性与多面性，突显其在需要自适应运动技能的各种自主操作任务中的应用潜力。