The process of revising (or constructing) a policy at execution time -- known as decision-time planning -- has been key to achieving superhuman performance in perfect-information games like chess and Go. A recent line of work has extended decision-time planning to imperfect-information games, leading to superhuman performance in poker. However, these methods involve solving subgames whose sizes grow quickly in the amount of non-public information, making them unhelpful when the amount of non-public information is large. Motivated by this issue, we introduce an alternative framework for decision-time planning that is not based on solving subgames, but rather on update equivalence. In this update-equivalence framework, decision-time planning algorithms replicate the updates of last-iterate algorithms, which need not rely on public information. This facilitates scalability to games with large amounts of non-public information. Using this framework, we derive a provably sound search algorithm for fully cooperative games based on mirror descent and a search algorithm for adversarial games based on magnetic mirror descent. We validate the performance of these algorithms in cooperative and adversarial domains, notably in Hanabi, the standard benchmark for search in fully cooperative imperfect-information games. Here, our mirror descent approach exceeds or matches the performance of public information-based search while using two orders of magnitude less search time. This is the first instance of a non-public-information-based algorithm outperforming public-information-based approaches in a domain they have historically dominated.

翻译：决策时规划——即在执行阶段修正（或构建）策略的过程——一直是实现完美信息博弈（如国际象棋和围棋）超人表现的关键。近期研究将决策时规划扩展至不完美信息博弈，从而在扑克中达成超人性能。然而，这些方法需要求解规模随非公开信息量急剧增长的子博弈，因此在非公开信息量庞大时效果有限。针对这一问题，我们提出一种不依赖子博弈求解、而是基于更新等价的决策时规划新框架。在该更新等价框架中，决策时规划算法复现最后迭代算法的更新过程，无需依赖公开信息，从而提升算法在非公开信息量庞大博弈中的可扩展性。基于此框架，我们推导出完全合作博弈中基于镜像下降的可证明高效搜索算法，以及对抗博弈中基于磁镜像下降的搜索算法。我们在合作与对抗领域验证了算法性能，特别是在完全合作不完美信息博弈的基准测试汉拿比中，我们的镜像下降方法在搜索时间降低两个数量级的情况下，其性能超越或持平于基于公开信息的搜索方法。这是首个在传统上由公开信息方法主导的领域中，非公开信息算法超越公开信息方法的实例。