For solving zero-sum games involving non-transitivity, a useful approach is to maintain a policy population to approximate the Nash Equilibrium (NE). Previous studies have shown that the Policy Space Response Oracles (PSRO) algorithm is an effective framework for solving such games. However, current methods initialize a new policy from scratch or inherit a single historical policy in Best Response (BR), missing the opportunity to leverage past policies to generate a better BR. In this paper, we propose Fusion-PSRO, which employs Nash Policy Fusion to initialize a new policy for BR training. Nash Policy Fusion serves as an implicit guiding policy that starts exploration on the current Meta-NE, thus providing a closer approximation to BR. Moreover, it insightfully captures a weighted moving average of past policies, dynamically adjusting these weights based on the Meta-NE in each iteration. This cumulative process further enhances the policy population. Empirical results on classic benchmarks show that Fusion-PSRO achieves lower exploitability, thereby mitigating the shortcomings of previous research on policy initialization in BR.

翻译：为解决涉及非传递性的零和博弈，一种有效方法是维护一个策略种群以近似纳什均衡（NE）。先前研究表明，策略空间响应预言机（PSRO）算法是求解此类博弈的有效框架。然而，现有方法在最佳响应（BR）中要么从零初始化新策略，要么仅继承单一历史策略，错失了利用过往策略生成更优BR的机会。本文提出Fusion-PSRO，该方法采用纳什策略融合来初始化用于BR训练的新策略。纳什策略融合作为一种隐式引导策略，从当前元纳什均衡开始探索，从而提供对BR的更接近近似。此外，该方法巧妙地捕捉了过往策略的加权移动平均，并依据每轮迭代的元纳什均衡动态调整这些权重。这种累积过程进一步增强了策略种群。经典基准测试的实验结果表明，Fusion-PSRO实现了更低的可利用性，从而改善了先前研究中关于BR策略初始化的不足。