Deep reinforcement learning has recently seen huge success across multiple areas in the robotics domain. Owing to the limitations of gathering real-world data, i.e., sample inefficiency and the cost of collecting it, simulation environments are utilized for training the different agents. This not only aids in providing a potentially infinite data source, but also alleviates safety concerns with real robots. Nonetheless, the gap between the simulated and real worlds degrades the performance of the policies once the models are transferred into real robots. Multiple research efforts are therefore now being directed towards closing this sim-to-real gap and accomplish more efficient policy transfer. Recent years have seen the emergence of multiple methods applicable to different domains, but there is a lack, to the best of our knowledge, of a comprehensive review summarizing and putting into context the different methods. In this survey paper, we cover the fundamental background behind sim-to-real transfer in deep reinforcement learning and overview the main methods being utilized at the moment: domain randomization, domain adaptation, imitation learning, meta-learning and knowledge distillation. We categorize some of the most relevant recent works, and outline the main application scenarios. Finally, we discuss the main opportunities and challenges of the different approaches and point to the most promising directions.

翻译：由于收集真实世界数据(即抽样效率低下和收集数据的成本)的局限性,模拟环境被用于培训不同的代理商。这不仅有助于提供潜在的无限数据源,而且减轻了对真实机器人的安全关切。然而,模拟世界与真实世界之间的差距使模型一旦转移到真正的机器人,政策绩效就会下降。因此,目前正在开展多项研究努力,以弥合这种模拟到真实的差距,并实现更有效的政策转移。近年来出现了适用于不同领域的多种方法,但根据我们的知识,我们缺乏对不同方法的全面审查,总结和介绍这些方法。在本调查文件中,我们探讨了在深度强化学习中进行模拟到真实的转让背后的基本背景,并概述了当前使用的主要方法:域随机化、域适应、模仿学习、元学习和知识蒸馏。我们将最近一些最相关的工作分类,并概述了主要应用方案的方向。最后,我们讨论了各种机会和最有希望的方法。