Effective exploration remains a key challenge in RL, especially with non-stationary rewards or high-dimensional policies. We introduce ARISE, a lightweight framework that enhances reinforcement learning by augmenting standard policy-gradient methods with a compact swarm-based exploration layer. ARISE blends policy actions with particle-driven proposals, where each particle represents a candidate policy trajectory sampled in the action space, and modulates exploration adaptively using reward-variance cues. While easy benchmarks exhibit only slight improvements (e.g., +0.7% on CartPole-v1), ARISE yields substantial gains on more challenging tasks, including +46% on LunarLander-v3 and +22% on Hopper-v4, while preserving stability on Walker2d and Ant. Under non-stationary reward shifts, ARISE provides marked robustness advantages, outperforming PPO by +75 points on CartPole and improving LunarLander accordingly. Ablation studies confirm that both the swarm component and the adaptive mechanism contribute to the performance. Overall, ARISE offers a simple, architecture-agnostic route to more exploratory and resilient RL agents without altering core algorithmic structures.

翻译：有效探索仍然是强化学习中的一个关键挑战，尤其是在面对非平稳奖励或高维策略时。本文提出了ARISE，一个轻量级框架，它通过向标准策略梯度方法添加一个紧凑的基于群体的探索层来增强强化学习。ARISE将策略动作与粒子驱动的提议相融合，其中每个粒子代表在动作空间中采样的候选策略轨迹，并利用奖励方差线索自适应地调节探索。虽然在简单基准测试中仅显示出轻微改进（例如，CartPole-v1上+0.7%），但ARISE在更具挑战性的任务上带来了显著增益，包括LunarLander-v3上+46%和Hopper-v4上+22%，同时在Walker2d和Ant上保持了稳定性。在非平稳奖励变化下，ARISE提供了显著的鲁棒性优势，在CartPole上优于PPO +75分，并在LunarLander上相应改进。消融研究证实，群体组件和自适应机制都对性能有所贡献。总体而言，ARISE提供了一种简单、架构无关的途径，无需改变核心算法结构，即可获得更具探索性和适应性的强化学习智能体。