Hindsight experience replay (HER) is well-known to accelerate goal-based reinforcement learning (RL). While HER is generally applied to off-policy RL algorithms, we previously showed that HER can also accelerate on-policy algorithms, such as proximal policy optimization (PPO), for goal-based Predator-Prey environments. Here, we show that we can improve the previous PPO-HER algorithm by selectively applying HER in a principled manner.

翻译：后见经验回放（HER）被广泛应用于加速基于目标的强化学习（RL）。虽然HER通常应用于离策略RL算法，但我们先前的研究表明，HER同样可以加速基于目标的捕食者-猎物环境中近端策略优化（PPO）等在线策略算法。本文提出通过原则性地选择性应用HER来改进先前的PPO-HER算法。