Building agents based on tree-search planning capabilities with learned models has achieved remarkable success in classic decision-making problems, such as Go and Atari. However, it has been deemed challenging or even infeasible to extend Monte Carlo Tree Search (MCTS) based algorithms to diverse real-world applications, especially when these environments involve complex action spaces and significant simulation costs, or inherent stochasticity. In this work, we introduce LightZero, the first unified benchmark for deploying MCTS/MuZero in general sequential decision scenarios. Specificially, we summarize the most critical challenges in designing a general MCTS-style decision-making solver, then decompose the tightly-coupled algorithm and system design of tree-search RL methods into distinct sub-modules. By incorporating more appropriate exploration and optimization strategies, we can significantly enhance these sub-modules and construct powerful LightZero agents to tackle tasks across a wide range of domains, such as board games, Atari, MuJoCo, MiniGrid and GoBigger. Detailed benchmark results reveal the significant potential of such methods in building scalable and efficient decision intelligence. The code is available as part of OpenDILab at https://github.com/opendilab/LightZero.

翻译：基于学习模型与树搜索规划能力的智能体构建已在围棋、Atari等经典决策问题中取得显著成功。然而，将基于蒙特卡洛树搜索（MCTS）的算法扩展到多样化的实际应用场景仍被视为具有挑战性甚至不可行的，尤其是当这些环境涉及复杂动作空间、高昂模拟成本或固有随机性时。本文提出LightZero——首个面向通用序贯决策场景部署MCTS/MuZero的统一基准。具体而言，我们总结了设计通用MCTS型决策求解器面临的最关键挑战，将树搜索强化学习方法的紧耦合算法与系统设计解耦为独立子模块。通过引入更恰当的探索与优化策略，我们显著增强这些子模块，并构建出强大的LightZero智能体，以应对棋盘游戏、Atari、MuJoCo、MiniGrid及GoBigger等多领域任务。详细的基准测试结果揭示了这类方法在构建可扩展高效决策智能方面的巨大潜力。相关代码已作为OpenDILab的一部分发布于https://github.com/opendilab/LightZero。