The Dreamer algorithm has recently obtained remarkable performance across diverse environment domains by training powerful agents with simulated trajectories. However, the compressed nature of its world model's latent space can result in the loss of crucial information, negatively affecting the agent's performance. Recent approaches, such as $Δ$-IRIS and DIAMOND, address this limitation by training more accurate world models. However, these methods require training agents directly from pixels, which reduces training efficiency and prevents the agent from benefiting from the inner representations learned by the world model. In this work, we propose an alternative approach to world modeling that is both accurate and efficient. We introduce EMERALD (Efficient MaskEd latent tRAnsformer worLD model), a world model using a spatial latent state with MaskGIT predictions to generate accurate trajectories in latent space and improve the agent performance. On the Crafter benchmark, EMERALD achieves new state-of-the-art performance, becoming the first method to surpass human experts performance within 10M environment steps. Our method also succeeds to unlock all 22 Crafter achievements at least once during evaluation.

翻译：Dreamer算法近期通过使用模拟轨迹训练强大智能体，在多样化环境领域中取得了显著性能。然而，其世界模型潜在空间的压缩特性可能导致关键信息丢失，进而对智能体性能产生负面影响。近期提出的方法（如$Δ$-IRIS和DIAMOND）通过训练更精确的世界模型来解决这一局限性。但这些方法需要直接从像素训练智能体，这会降低训练效率，并阻碍智能体从世界模型学习到的内部表征中获益。本研究提出一种兼具精确性与高效性的世界建模替代方案。我们引入EMERALD（高效掩码潜在Transformer世界模型），该模型采用空间潜在状态与MaskGIT预测机制，在潜在空间中生成精确轨迹并提升智能体性能。在Crafter基准测试中，EMERALD实现了新的最先进性能，成为首个在1000万环境步数内超越人类专家水平的方法。我们的方法在评估期间还成功解锁了全部22项Crafter成就至少各一次。