Semantic representations in higher sensory cortices form the basis for robust, yet flexible behavior. These representations are acquired over the course of development in an unsupervised fashion and continuously maintained over an organism's lifespan. Predictive learning theories propose that these representations emerge from predicting or reconstructing sensory inputs. However, brains are known to generate virtual experiences, such as during imagination and dreaming, that go beyond previously experienced inputs. Here, we suggest that virtual experiences may be just as relevant as actual sensory inputs in shaping cortical representations. In particular, we discuss two complementary learning principles that organize representations through the generation of virtual experiences. First, "adversarial dreaming" proposes that creative dreams support a cortical implementation of adversarial learning in which feedback and feedforward pathways engage in a productive game of trying to fool each other. Second, "contrastive dreaming" proposes that the invariance of neuronal representations to irrelevant factors of variation is acquired by trying to map similar virtual experiences together via a contrastive learning process. These principles are compatible with known cortical structure and dynamics and the phenomenology of sleep thus providing promising directions to explain cortical learning beyond the classical predictive learning paradigm.

翻译：高级感觉皮层中的语义表征构成了稳健而灵活行为的基础。这些表征是在发育过程中以无监督方式获得，并在生物体整个生命周期中持续维持。预测性学习理论认为，这些表征源于对感觉输入的预测或重构。然而，已知大脑会产生超越先前输入经验的虚拟体验，例如在想象和做梦期间。本文提出，虚拟体验在塑造皮层表征方面可能与实际感觉输入同样重要。具体而言，我们讨论两种通过生成虚拟体验来组织表征的互补学习原则。第一，"对抗性做梦"（adversarial dreaming）提出，创造性梦境支持对抗性学习的皮层实现，其中反馈和前馈通路参与一场试图相互愚弄的生产性博弈。第二，"对比性做梦"（contrastive dreaming）提出，神经表征对无关变异因素的不变性是通过对比学习过程将相似虚拟体验映射在一起而获得的。这些原则与已知的皮层结构和动力学以及睡眠现象学相容，从而为解释超越经典预测性学习范式的皮层学习提供了有前景的方向。