Unlike most reinforcement learning agents which require an unrealistic amount of environment interactions to learn a new behaviour, humans excel at learning quickly by merely observing and imitating others. This ability highly depends on the fact that humans have a model of their own embodiment that allows them to infer the most likely actions that led to the observed behaviour. In this paper, we propose Action Inference by Maximising Evidence (AIME) to replicate this behaviour using world models. AIME consists of two distinct phases. In the first phase, the agent learns a world model from its past experience to understand its own body by maximising the ELBO. While in the second phase, the agent is given some observation-only demonstrations of an expert performing a novel task and tries to imitate the expert's behaviour. AIME achieves this by defining a policy as an inference model and maximising the evidence of the demonstration under the policy and world model. Our method is "zero-shot" in the sense that it does not require further training for the world model or online interactions with the environment after given the demonstration. We empirically validate the zero-shot imitation performance of our method on the Walker and Cheetah embodiment of the DeepMind Control Suite and find it outperforms the state-of-the-art baselines. Code is available at: https://github.com/argmax-ai/aime.

翻译：与大多数需要与环境进行不切实际的大量交互才能学习新行为的强化学习智能体不同，人类仅通过观察和模仿他人就能快速学习，这得益于他们拥有自身具身模型，能够推断出最可能导致所观察行为的动作。本文提出通过最大化证据进行动作推断（AIME）方法，利用世界模型复现这一行为。AIME包含两个不同阶段：第一阶段，智能体通过最大化证据下界（ELBO），从过往经验中学习理解自身身体的世界模型；第二阶段，智能体获得专家执行新任务的仅观察演示（observation-only demonstrations），并尝试模仿专家行为——通过将策略定义为推断模型，并最大化该演示在策略与世界模型下的证据（evidence）。该方法属于“零样本”（zero-shot），即在获得演示后无需额外训练世界模型或与环境进行在线交互。我们在DeepMind控制套件（DeepMind Control Suite）的Walker和Cheetah具身上实验验证了零样本模仿性能，发现其优于当前最先进的基线方法。代码开源地址：https://github.com/argmax-ai/aime。