In combination with Reinforcement Learning, Monte-Carlo Tree Search has shown to outperform human grandmasters in games such as Chess, Shogi and Go with little to no prior domain knowledge. However, most classical use cases only feature up to two players. Scaling the search to an arbitrary number of players presents a computational challenge, especially if decisions have to be planned over a longer time horizon. In this work, we investigate techniques that transform general-sum multiplayer games into single-player and two-player games that consider other agents to act according to given opponent models. For our evaluation, we focus on the challenging Pommerman environment which involves partial observability, a long time horizon and sparse rewards. In combination with our search methods, we investigate the phenomena of opponent modeling using heuristics and self-play. Overall, we demonstrate the effectiveness of our multiplayer search variants both in a supervised learning and reinforcement learning setting.

翻译：蒙特卡洛树搜索结合强化学习后，在象棋、将棋和围棋等游戏中，无需或仅需极少的领域先验知识，即可超越人类顶尖棋手。然而，大多数经典应用场景仅涉及最多两名玩家。将搜索扩展到任意数量玩家具有计算挑战性，尤其是当决策需要在较长的时间范围内规划时。本研究探讨了将一般和多人游戏转化为单人及双人游戏的技术，其中其他智能体将依据给定的对手模型行动。我们选择具有部分可观测性、长时程和稀疏奖励挑战性的Pommerman环境进行评估。结合所提出的搜索方法，我们利用启发式方法和自博弈研究了对手建模现象。总体而言，我们在监督学习和强化学习场景中均验证了多人搜索变体的有效性。