Simulation provides a safe and efficient way to generate useful data for learning complex robotic tasks. However, matching simulation and real-world dynamics can be quite challenging, especially for systems that have a large number of unobserved or unmeasurable parameters, which may lie in the robot dynamics itself or in the environment with which the robot interacts. We introduce a novel approach to tackle such a sim-to-real problem by developing policies capable of adapting to new environments, in a zero-shot manner. Key to our approach is an error-aware policy (EAP) that is explicitly made aware of the effect of unobservable factors during training. An EAP takes as input the predicted future state error in the target environment, which is provided by an error-prediction function, simultaneously trained with the EAP. We validate our approach on an assistive walking device trained to help the human user recover from external pushes. We show that a trained EAP for a hip-torque assistive device can be transferred to different human agents with unseen biomechanical characteristics. In addition, we show that our method can be applied to other standard RL control tasks.

翻译：模拟为生成有用的数据以学习复杂的机器人任务提供了一个安全有效的模拟方法。 但是,匹配模拟和真实世界动态可能具有相当大的挑战性,特别是对于拥有大量不可观测或不可测量参数的系统来说,这些参数可能存在于机器人动态本身或机器人与之互动的环境中。我们采用一种新颖的方法,通过制定能够适应新环境的政策,以零弹方式解决这种模拟到现实的问题。我们的方法的关键在于一种对错误的认识政策(EAP),该政策在培训期间明确意识到不可观测因素的影响。一个EAP将目标环境中预测的未来状态错误作为输入,该错误控制功能提供,同时与EAP一起培训。我们验证了我们关于帮助人类用户从外部推力中恢复过来的辅助行走装置的方法。我们表明,经过培训的峰式辅助装置的EAP可以转让给具有看不见生物机械特性的不同人类物剂。此外,我们还表明,我们的方法可以应用于其他标准的 RL 控制任务。