Informational parsimony -- i.e., using the minimal information required for a task, -- provides a useful inductive bias for learning representations that achieve better generalization by being robust to noise and spurious correlations. We propose information gating in the pixel space as a way to learn more parsimonious representations. Information gating works by learning masks that capture only the minimal information required to solve a given task. Intuitively, our models learn to identify which visual cues actually matter for a given task. We gate information using a differentiable parameterization of the signal-to-noise ratio, which can be applied to arbitrary values in a network, e.g.~masking out pixels at the input layer. We apply our approach, which we call InfoGating, to various objectives such as: multi-step forward and inverse dynamics, Q-learning, behavior cloning, and standard self-supervised tasks. Our experiments show that learning to identify and use minimal information can improve generalization in downstream tasks -- e.g., policies based on info-gated images are considerably more robust to distracting/irrelevant visual features.

翻译：信息简约性——即仅使用任务所需的最小信息量——提供了一种有用的归纳偏置，用于学习对噪声和虚假关联具有鲁棒性、从而获得更好泛化能力的表示。我们提出在像素空间中使用信息门控来学习更简约的表示。信息门控通过学习掩码来捕获解决给定任务所需的最小信息量。直观上，我们的模型学习识别哪些视觉线索对特定任务真正重要。我们利用信噪比的可微参数化来门控信息，该方法可应用于网络中的任意数值，例如在输入层屏蔽像素。我们将此方法称为InfoGating，并将其应用于多种目标，例如：多步前向与逆动力学、Q学习、行为克隆以及标准的自监督任务。实验表明，学习识别并使用最小信息可以提升下游任务的泛化能力——例如，基于信息门控图像构建的策略对干扰/无关视觉特征具有显著更强的鲁棒性。