In recent years, reinforcement learning and imitation learning have shown great potential for controlling humanoid robots' motion. However, these methods typically create simulation environments and rewards for specific tasks, resulting in the requirements of multiple policies and limited capabilities for tackling complex and unknown tasks. To overcome these issues, we present a novel approach that combines adversarial imitation learning with large language models (LLMs). This innovative method enables the agent to learn reusable skills with a single policy and solve zero-shot tasks under the guidance of LLMs. In particular, we utilize the LLM as a strategic planner for applying previously learned skills to novel tasks through the comprehension of task-specific prompts. This empowers the robot to perform the specified actions in a sequence. To improve our model, we incorporate codebook-based vector quantization, allowing the agent to generate suitable actions in response to unseen textual commands from LLMs. Furthermore, we design general reward functions that consider the distinct motion features of humanoid robots, ensuring the agent imitates the motion data while maintaining goal orientation without additional guiding direction approaches or policies. To the best of our knowledge, this is the first framework that controls humanoid robots using a single learning policy network and LLM as a planner. Extensive experiments demonstrate that our method exhibits efficient and adaptive ability in complicated motion tasks.

翻译：近年来，强化学习和模仿学习在控制类人机器人运动方面展现出巨大潜力。然而，这些方法通常需要针对特定任务构建仿真环境和奖励函数，导致需要多个策略，且处理复杂及未知任务的能力有限。为解决这些问题，我们提出了一种新颖方法，将对抗性模仿学习与大语言模型相结合。这种创新方法使智能体能够通过单一策略学习可复用技能，并在大语言模型引导下解决零样本任务。具体而言，我们利用大语言模型作为战略规划器，通过理解任务特定提示将先前学到的技能应用于新任务，从而使机器人能够按顺序执行指定动作。为改进模型，我们引入基于码本向量量化技术，使智能体能够对大语言模型生成的非文本指令生成合适动作。此外，我们设计了考虑类人机器人独特运动特征的通用奖励函数，确保智能体在模仿运动数据的同时保持目标导向性，无需额外导向方法或策略。据我们所知，这是首个利用单一学习策略网络与大语言模型规划器控制类人机器人的框架。大量实验证明，我们的方法在复杂运动任务中展现出高效且自适应的能力。