Deep reinforcement learning has recently achieved strong results in quadrupedal locomotion, yet policies trained in simulation often fail to transfer when the environment changes. Evolutionary reinforcement learning aims to address this limitation by combining gradient-based policy optimisation with population-driven exploration. This work evaluates four methods on a simulated walking task: DDPG, TD3, and two Cross-Entropy-based variants CEM-DDPG and CEM-TD3. All agents are trained on flat terrain and later tested both on this domain and on a rough terrain not encountered during training. TD3 performs best among the standard deep RL baselines on flat ground with a mean reward of 5927.26, while CEM-TD3 achieves the highest rewards overall during training and evaluation 17611.41. Under the rough-terrain transfer test, performance of the deep RL methods drops sharply. DDPG achieves -1016.32 and TD3 achieves -99.73, whereas the evolutionary variants retain much of their capability. CEM-TD3 records the strongest transfer performance with a mean reward of 19574.33. These findings suggest that incorporating evolutionary search can reduce overfitting and improve policy robustness in locomotion tasks, particularly when deployment conditions differ from those seen during training.

翻译：深度强化学习最近在四足运动方面取得了显著成果，但模拟训练的策略在环境变化时往往难以迁移。进化强化学习旨在通过将基于梯度的策略优化与群体驱动的探索相结合来解决这一局限性。本研究在模拟行走任务上评估了四种方法：DDPG、TD3，以及两种基于交叉熵的变体CEM-DDPG和CEM-TD3。所有智能体在平坦地形上训练，随后在该领域及训练中未遇到的崎岖地形上进行测试。在平坦地面上，TD3在标准深度强化学习基线中表现最佳，平均奖励为5927.26，而CEM-TD3在训练和评估中均获得最高总体奖励17611.41。在崎岖地形迁移测试下，深度强化学习方法的性能急剧下降。DDPG达到-1016.32，TD3达到-99.73，而进化变体则保留了大部分能力。CEM-TD3记录了最强的迁移性能，平均奖励为19574.33。这些发现表明，融入进化搜索可以减少过拟合并提高运动任务中的策略鲁棒性，特别是在部署条件与训练时不同的情况下。