Simulation to Real-World Transfer allows affordable and fast training of learning-based robots for manipulation tasks using Deep Reinforcement Learning methods. Currently, Sim2Real uses Asymmetric Actor-Critic approaches to reduce the rich idealized features in simulation to the accessible ones in the real world. However, the feature reduction from the simulation to the real world is conducted through an empirically defined one-step curtail. Small feature reduction does not sufficiently remove the actor's features, which may still cause difficulty setting up the physical system, while large feature reduction may cause difficulty and inefficiency in training. To address this issue, we proposed Curriculum-based Sensing Reduction to enable the actor to start with the same rich feature space as the critic and then get rid of the hard-to-extract features step-by-step for higher training performance and better adaptation for real-world feature space. The reduced features are replaced with random signals from a Deep Random Generator to remove the dependency between the output and the removed features and avoid creating new dependencies. The methods are evaluated on the Allegro robot hand in a real-world in-hand manipulation task. The results show that our methods have faster training and higher task performance than baselines and can solve real-world tasks when selected tactile features are reduced.

翻译：仿真到现实迁移通过深度强化学习方法，能够以较低成本高效训练基于学习的机器人完成操作任务。当前，Sim2Real采用非对称Actor-Critic方法，将仿真中丰富的理想化特征缩减为现实世界中可获取的特征。然而，从仿真到现实的特征缩减是通过经验定义的单步削减实现的。特征缩减幅度过小无法充分去除Actor依赖的特征，仍可能导致物理系统搭建困难；而特征缩减幅度过大则可能造成训练困难与效率低下。为解决该问题，我们提出基于课程的感觉缩减方法，使Actor初始阶段保持与Critic相同的丰富特征空间，随后逐步剔除难以提取的特征，从而提升训练性能并更适应现实世界的特征空间。被缩减的特征由深度随机生成器产生的随机信号替代，以消除输出与被移除特征之间的依赖关系，并避免产生新依赖。该方法在Allegro机器人手上进行了真实世界手内操作任务评估。结果表明，与基线方法相比，我们的方法具有更快的训练速度和更高的任务性能，且能在减少选定触觉特征的条件下解决现实任务。