Although reinforcement learning methods can achieve impressive results in simulation, the real world presents two major challenges: generating samples is exceedingly expensive, and unexpected perturbations can cause proficient but narrowly-learned policies to fail at test time. In this work, we propose to learn how to quickly and effectively adapt online to new situations as well as to perturbations. To enable sample-efficient meta-learning, we consider learning online adaptation in the context of model-based reinforcement learning. Our approach trains a global model such that, when combined with recent data, the model can be be rapidly adapted to the local context. Our experiments demonstrate that our approach can enable simulated agents to adapt their behavior online to novel terrains, to a crippled leg, and in highly-dynamic environments.

翻译：虽然强化学习方法在模拟中可以取得令人印象深刻的成果,但真实世界却提出了两大挑战:生成样本费用极高,而意外的扰动可能导致在测试时巧妙但收效狭窄的政策失败。 在这项工作中,我们提议学习如何迅速和有效地在网上适应新形势和扰动。为了能够实现样本效率高的元学习,我们考虑在基于模型的强化学习中学习在线适应。我们的方法培训了一个全球模型,这样,如果与最近的数据相结合,模型可以迅速适应当地环境。我们的实验表明,我们的方法可以让模拟代理人在网上将其行为适应新奇地形、残缺的腿和高动态环境中。