Imagination in world models is crucial for enabling agents to learn long-horizon policy in a sample-efficient manner. Existing recurrent state-space model (RSSM)-based world models depend on single-step statistical inference to capture the environment dynamics, and, hence, they are unable to perform long-term imagination tasks due to the accumulation of prediction errors. Inspired by the dual-process theory of human cognition, we propose a novel dual-mind world model (DMWM) framework that integrates logical reasoning to enable imagination with logical consistency. DMWM is composed of two components: an RSSM-based System 1 (RSSM-S1) component that handles state transitions in an intuitive manner and a logic-integrated neural network-based System 2 (LINN-S2) component that guides the imagination process through hierarchical deep logical reasoning. The inter-system feedback mechanism is designed to ensure that the imagination process follows the logical rules of the real environment. The proposed framework is evaluated on benchmark tasks that require long-term planning from the DMControl suite. Extensive experimental results demonstrate that the proposed framework yields significant improvements in terms of logical coherence, trial efficiency, data efficiency and long-term imagination over the state-of-the-art world models.

翻译：在世界模型中，想象力对于使智能体能够以样本高效的方式学习长时程策略至关重要。现有的基于循环状态空间模型（RSSM）的世界模型依赖单步统计推断来捕捉环境动态，因此由于预测误差的累积而无法执行长期想象任务。受人类认知的双过程理论启发，我们提出了一种新颖的双心智世界模型（DMWM）框架，该框架集成了逻辑推理以实现具有逻辑一致性的想象。DMWM由两个组件构成：一个基于RSSM的系统1（RSSM-S1）组件，以直觉方式处理状态转移；以及一个基于逻辑集成神经网络的系统2（LINN-S2）组件，通过分层深度逻辑推理来指导想象过程。系统间反馈机制旨在确保想象过程遵循真实环境的逻辑规则。所提出的框架在DMControl套件中需要长期规划的基准任务上进行了评估。大量的实验结果表明，与最先进的世界模型相比，所提出的框架在逻辑一致性、试验效率、数据效率和长期想象力方面均取得了显著提升。