Humanoid robots, designed to operate in human-centric environments, serve as a fundamental platform for a broad range of tasks. Although humanoid robots have been extensively studied for decades, a majority of existing humanoid robots still heavily rely on complex modular frameworks, leading to inflexibility and potential compounded errors from independent sensing, planning, and acting components. In response, we propose an end-to-end humanoid sense-plan-act walking system, enabling vision-based obstacle avoidance and footstep planning for whole body balancing simultaneously. We designed two imperative learning (IL)-based bilevel optimizations for model-predictive step planning and whole body balancing, respectively, to achieve self-supervised learning for humanoid robot walking. This enables the robot to learn from arbitrary unlabeled data, improving its adaptability and generalization capabilities. We refer to our method as iWalker and demonstrate its effectiveness in both simulated and real-world environments, representing a significant advancement toward autonomous humanoid robots.

翻译：人形机器人被设计用于以人为中心的环境中运行，是广泛任务的基础平台。尽管人形机器人已被广泛研究数十年，但大多数现有人形机器人仍严重依赖复杂的模块化框架，导致灵活性不足，且独立的感知、规划与执行组件可能产生复合误差。为此，我们提出了一种端到端的人形机器人感知-规划-执行行走系统，能够同时实现基于视觉的避障与面向全身平衡的步态规划。我们分别针对模型预测步态规划和全身平衡设计了两层基于指令式学习（IL）的双层优化方法，以实现人形机器人行走的自监督学习。这使得机器人能够从任意未标记数据中学习，提升其适应性和泛化能力。我们将该方法称为iWalker，并在仿真和真实环境中验证了其有效性，代表了向自主人形机器人发展的重要进展。