Model-based reinforcement learning (MBRL) is recognized with the potential to be significantly more sample-efficient than model-free RL. How an accurate model can be developed automatically and efficiently from raw sensory inputs (such as images), especially for complex environments and tasks, is a challenging problem that hinders the broad application of MBRL in the real world. In this work, we propose a sensing-aware model-based reinforcement learning system called SAM-RL. Leveraging the differentiable physics-based simulation and rendering, SAM-RL automatically updates the model by comparing rendered images with real raw images and produces the policy efficiently. With the sensing-aware learning pipeline, SAM-RL allows a robot to select an informative viewpoint to monitor the task process. We apply our framework to real world experiments for accomplishing three manipulation tasks: robotic assembly, tool manipulation, and deformable object manipulation. We demonstrate the effectiveness of SAM-RL via extensive experiments. Videos are available on our project webpage at https://sites.google.com/view/rss-sam-rl.

翻译：基于模型的强化学习（MBRL）因其潜在的高于无模型强化学习的样本效率而备受关注。如何从原始感官输入（如图像）中自动且高效地开发精准模型，尤其针对复杂环境与任务，成为制约MBRL在现实世界中广泛应用的挑战性问题。本文提出一种名为SAM-RL的感知感知模型驱动强化学习系统。SAM-RL利用可微物理模拟与渲染技术，通过比较渲染图像与实际原始图像自动更新模型，并高效生成策略。借助感知感知学习流水线，SAM-RL使机器人能够选择信息丰富的视角来监控任务进程。我们将该框架应用于真实世界实验，完成三项操作任务：机器人组装、工具操作及可变形物体操作。通过大量实验验证了SAM-RL的有效性。相关视频见项目网页：https://sites.google.com/view/rss-sam-rl。