Semantic representations in higher sensory cortices form the basis for robust, yet flexible behavior. These representations are acquired over the course of development in an unsupervised fashion and continuously maintained over an organism's lifespan. Predictive learning theories propose that these representations emerge from predicting or reconstructing sensory inputs. However, brains are known to generate virtual experiences, such as during imagination and dreaming, that go beyond previously experienced inputs. Here, we suggest that virtual experiences may be just as relevant as actual sensory inputs in shaping cortical representations.In particular, we discuss two complementary learning principles that organize representations through the generation of virtual experiences. First, "adversarial dreaming" proposes that creative dreams support a cortical implementation of adversarial learning in which feedback and feedforward pathways engage in a productive game of trying to fool each other. Second, "contrastive dreaming" proposes that the invariance of neuronal representations to irrelevant factors of variation is acquired by trying to map similar virtual experiences together via a contrastive learning process. These principles are compatible with known cortical structure and dynamics and the phenomenology of sleep thus providing promising directions to explain cortical learning beyond the classical predictive learning paradigm.

翻译：高级感觉皮层中的语义表征构成了稳健而灵活行为的基础。这些表征以无监督方式在发育过程中习得，并在生物体整个生命周期中持续维持。预测学习理论提出，这些表征源于对感觉输入的预测或重建。然而，已知大脑能够生成超出既往输入经验的虚拟体验，例如在想象和梦境中。在此，我们提出虚拟体验在塑造皮层表征方面可能与实际感觉输入同等重要。具体而言，我们讨论了两种通过生成虚拟体验来组织表征的互补学习原则。其一，“对抗性做梦”提出创造性梦境支持皮层实现对抗性学习，其中反馈通路与前馈通路参与一场相互欺骗的生产性博弈。其二，“对比性做梦”提出神经元表征对无关变量因素的不变性，是通过对比学习过程试图将相似虚拟体验映射在一起而获得的。这些原则与已知的皮层结构和动力学以及睡眠现象学兼容，从而为解释超越经典预测学习范式的皮层学习提供了有前景的方向。