Levin Tree Search (LTS) is a search algorithm that makes use of a policy (a probability distribution over actions) and comes with a theoretical guarantee on the number of expansions before reaching a goal node, depending on the quality of the policy. This guarantee can be used as a loss function, which we call the LTS loss, to optimize neural networks representing the policy (LTS+NN). In this work we show that the neural network can be substituted with parameterized context models originating from the online compression literature (LTS+CM). We show that the LTS loss is convex under this new model, which allows for using standard convex optimization tools, and obtain convergence guarantees to the optimal parameters in an online setting for a given set of solution trajectories -- guarantees that cannot be provided for neural networks. The new LTS+CM algorithm compares favorably against LTS+NN on several benchmarks: Sokoban (Boxoban), The Witness, and the 24-Sliding Tile puzzle (STP). The difference is particularly large on STP, where LTS+NN fails to solve most of the test instances while LTS+CM solves each test instance in a fraction of a second. Furthermore, we show that LTS+CM is able to learn a policy that solves the Rubik's cube in only a few hundred expansions, which considerably improves upon previous machine learning techniques.

翻译：莱文树搜索（LTS）是一种利用策略（动作上的概率分布）的搜索算法，其理论保证在到达目标节点前的扩展次数取决于策略的质量。该保证可被用作损失函数（我们称之为LTS损失），以优化表示策略的神经网络（LTS+NN）。本工作中，我们证明神经网络可被替换为源自在线压缩文献的参数化上下文模型（LTS+CM）。我们证明在此新模型下LTS损失是凸函数，从而能够使用标准凸优化工具，并在给定解轨迹集的在线场景中获得向最优参数收敛的保证——这是神经网络无法提供的理论保证。新算法LTS+CM在多个基准测试中表现优于LTS+NN：推箱子（Boxoban）、见证者游戏（The Witness）以及24数码滑块拼图（STP）。在STP上的差异尤为显著，LTS+NN无法解决大多数测试实例，而LTS+CM仅需不到一秒即可解决每个测试实例。此外，我们证明LTS+CM能够学习仅需数百次扩展即可解决魔方的策略，这显著超越了以往的机器学习技术。