Developing systems that can synthesize natural and life-like motions for simulated characters has long been a focus for computer animation. But in order for these systems to be useful for downstream applications, they need not only produce high-quality motions, but must also provide an accessible and versatile interface through which users can direct a character's behaviors. Natural language provides a simple-to-use and expressive medium for specifying a user's intent. Recent breakthroughs in natural language processing (NLP) have demonstrated effective use of language-based interfaces for applications such as image generation and program synthesis. In this work, we present PADL, which leverages recent innovations in NLP in order to take steps towards developing language-directed controllers for physics-based character animation. PADL allows users to issue natural language commands for specifying both high-level tasks and low-level skills that a character should perform. We present an adversarial imitation learning approach for training policies to map high-level language commands to low-level controls that enable a character to perform the desired task and skill specified by a user's commands. Furthermore, we propose a multi-task aggregation method that leverages a language-based multiple-choice question-answering approach to determine high-level task objectives from language commands. We show that our framework can be applied to effectively direct a simulated humanoid character to perform a diverse array of complex motor skills.

翻译：长期以来，开发能够为仿真角色合成自然逼真动作的系统一直是计算机动画领域的核心目标。然而，为使此类系统在后续应用中发挥作用，不仅需要生成高质量动作，还需提供便捷且多功能的交互接口，使用户能够指导角色的行为。自然语言作为一种易于使用且富有表现力的媒介，能够清晰表达用户意图。近年来自然语言处理（NLP）领域的突破性进展已证明，语言交互界面在图像生成、程序合成等应用中具有显著成效。本研究提出PADL框架，通过借鉴NLP领域的最新创新成果，向实现基于物理的角色动画语言控制器迈出关键一步。PADL允许用户通过自然语言指令指定角色应执行的高层级任务与低层级技能。我们提出一种对抗式模仿学习方法，用于训练策略将高层语言指令映射至低层控制信号，使角色能够执行用户指令指定的目标任务与技能。此外，我们提出一种基于语言多项选择问答机制的多任务聚合方法，可从语言指令中推导高层任务目标。实验表明，该框架可有效引导仿真人形角色执行多样化的复杂运动技能。