We introduce the $\gamma$-model, a predictive model of environment dynamics with an infinite probabilistic horizon. Replacing standard single-step models with $\gamma$-models leads to generalizations of the procedures central to model-based control, including the model rollout and model-based value estimation. The $\gamma$-model, trained with a generative reinterpretation of temporal difference learning, is a natural continuous analogue of the successor representation and a hybrid between model-free and model-based mechanisms. Like a value function, it contains information about the long-term future; like a standard predictive model, it is independent of task reward. We instantiate the $\gamma$-model as both a generative adversarial network and normalizing flow, discuss how its training reflects an inescapable tradeoff between training-time and testing-time compounding errors, and empirically investigate its utility for prediction and control.

翻译：我们引入了$gamma$模型,这是环境动态的预测模型,具有无限的概率地平线。用$gamma$模型取代标准的单步模型。用$gamma$模型取代标准的单步模型,导致基于模型的控制中心程序的概括化,包括模型推出和基于模型的价值估计。 $gamma$模型,经过对时间差异学习的基因重新解释培训,是继任代表的自然连续的类比,是无模型和基于模型的机制之间的混合。它与价值功能一样,包含关于长期未来的信息;与标准预测模型一样,它独立于任务奖励。我们将$gamma$模型作为基因化的对抗网络和正常的流动,我们讨论其培训如何反映培训时间和测试时间复合错误之间的不可避免的权衡,以及实证地调查其对预测和控制的效用。