Recent breakthroughs in Artificial Intelligence have shown that the combination of tree-based planning with deep learning can lead to superior performance. We present Adaptive Entropy Tree Search (ANTS) - a novel algorithm combining planning and learning in the maximum entropy paradigm. Through a comprehensive suite of experiments on the Atari benchmark we show that ANTS significantly outperforms PUCT, the planning component of the state-of-the-art AlphaZero system. ANTS builds upon recent work on maximum entropy planning methods - which however, as we show, fail in combination with learning. ANTS resolves this issue to reach state-of-the-art performance. We further find that ANTS exhibits superior robustness to different hyperparameter choices, compared to the previous algorithms. We believe that the high performance and robustness of ANTS can bring tree search planning one step closer to wide practical adoption.

翻译：人工智能领域的最新突破表明，将基于树的规划与深度学习相结合可带来卓越性能。我们提出自适应熵树搜索（ANTS）——一种在最大熵范式下融合规划与学习的新型算法。通过在Atari基准测试上的综合实验，我们证明ANTS显著优于当前最先进AlphaZero系统的规划组件PUCT。ANTS基于近期最大熵规划方法的研究——然而我们表明，这些方法在与学习结合时存在失效问题。ANTS解决了这一难题，实现了最先进的性能。我们进一步发现，与先前算法相比，ANTS对不同超参数选择展现出更优越的鲁棒性。我们相信ANTS的高性能与鲁棒性将推动树搜索规划向广泛实际应用迈出关键一步。