The ability to separate signal from noise, and reason with clean abstractions, is critical to intelligence. With this ability, humans can efficiently perform real world tasks without considering all possible nuisance factors.How can artificial agents do the same? What kind of information can agents safely discard as noises? In this work, we categorize information out in the wild into four types based on controllability and relation with reward, and formulate useful information as that which is both controllable and reward-relevant. This framework clarifies the kinds information removed by various prior work on representation learning in reinforcement learning (RL), and leads to our proposed approach of learning a Denoised MDP that explicitly factors out certain noise distractors. Extensive experiments on variants of DeepMind Control Suite and RoboDesk demonstrate superior performance of our denoised world model over using raw observations alone, and over prior works, across policy optimization control tasks as well as the non-control task of joint position regression.

翻译：将信号从噪音和理性中分离出来的能力以及清晰的抽象感对智能至关重要。 有了这种能力, 人类就可以在不考虑所有可能的麻烦因素的情况下高效地完成真实的世界任务。 人工剂如何能做到同样呢? 什么样的信息能将安全丢弃作为噪音? 在这项工作中,我们将野生信息分为四种类型, 其依据是可控性和与奖赏的关系, 并编制有用的信息, 因为它既是可控的, 也与奖赏相关。 这个框架澄清了先前在强化学习中的各种代言学习工作( RL ) 所删除的各类信息, 并导致我们提议的学习一个分解式的 MDP 的方法, 明确将某些噪声分散器作为因素。 关于深海控制套件和 RoboDesk 的变体的广泛实验展示了我们已分解的世界模型的优异性表现, 仅使用原始观测, 并超越先前的工作, 跨越政策优化控制任务以及联合定位回归的非控制任务。