This review addresses the problem of learning abstract representations of the measurement data in the context of Deep Reinforcement Learning (DRL). While the data are often ambiguous, high-dimensional, and complex to interpret, many dynamical systems can be effectively described by a low-dimensional set of state variables. Discovering these state variables from the data is a crucial aspect for (i) improving the data efficiency, robustness, and generalization of DRL methods, (ii) tackling the curse of dimensionality, and (iii) bringing interpretability and insights into black-box DRL. This review provides a comprehensive and complete overview of unsupervised representation learning in DRL by describing the main Deep Learning tools used for learning representations of the world, providing a systematic view of the method and principles, summarizing applications, benchmarks and evaluation strategies, and discussing open challenges and future directions.

翻译：本综述探讨了在深度强化学习（DRL）背景下，如何从测量数据中学习抽象表征的问题。尽管数据往往存在歧义性、高维性且解释复杂，但许多动态系统可通过低维状态变量集进行有效描述。从数据中发掘这些状态变量对于以下方面至关重要：（i）提升DRL方法的数据效率、鲁棒性与泛化能力，（ii）应对维数灾难，（iii）为黑箱DRL带来可解释性与深层洞察。本文通过阐述用于学习世界表征的主要深度学习工具、系统梳理方法与原理框架、总结应用场景、基准测试与评估策略，并探讨开放挑战与未来方向，对DRL中的无监督表征学习提供了全面系统的概述。