Humanoid robots hold great potential for diverse interactions and daily service tasks within human-centered environments, necessitating controllers that seamlessly integrate precise locomotion with dexterous manipulation. However, most existing whole-body controllers lack exteroceptive awareness of the surrounding environment, rendering them insufficient for stable task execution in complex, unstructured scenarios.To address this challenge, we propose PILOT, a unified single-stage reinforcement learning (RL) framework tailored for perceptive loco-manipulation, which synergizes perceptive locomotion and expansive whole-body control within a single policy. To enhance terrain awareness and ensure precise foot placement, we design a cross-modal context encoder that fuses prediction-based proprioceptive features with attention-based perceptive representations. Furthermore, we introduce a Mixture-of-Experts (MoE) policy architecture to coordinate diverse motor skills, facilitating better specialization across distinct motion patterns. Extensive experiments in both simulation and on the physical Unitree G1 humanoid robot validate the efficacy of our framework. PILOT demonstrates superior stability, command tracking precision, and terrain traversability compared to existing baselines. These results highlight its potential to serve as a robust, foundational low-level controller for loco-manipulation in unstructured scenes.

翻译：人形机器人在以人为中心的环境中具有巨大的交互和服务任务潜力，这需要控制器能够无缝集成精确的运动能力与灵巧的操作技能。然而，现有的大多数全身控制器缺乏对周围环境的外感知能力，导致其在复杂非结构化场景中难以稳定执行任务。为解决这一挑战，我们提出了PILOT，一个专为可感知运动操作设计的统一单阶段强化学习框架，它将感知运动与扩展的全身控制协同整合在单一策略中。为增强地形感知并确保精确的足部放置，我们设计了一个跨模态上下文编码器，融合基于预测的本体感知特征与基于注意力的感知表征。此外，我们引入了专家混合策略架构，以协调多种运动技能，促进不同运动模式间更好的专业化分工。在仿真和实体Unitree G1人形机器人上进行的大量实验验证了我们框架的有效性。与现有基线相比，PILOT在稳定性、指令跟踪精度和地形通过能力方面均表现出优越性能。这些结果凸显了其作为非结构化场景中运动操作的鲁棒基础低级控制器的潜力。