Reinforcement Learning is a powerful tool to model decision-making processes. However, it relies on an exploration-exploitation trade-off that remains an open challenge for many tasks. In this work, we study neighboring state-based, model-free exploration led by the intuition that, for an early-stage agent, considering actions derived from a bounded region of nearby states may lead to better actions when exploring. We propose two algorithms that choose exploratory actions based on a survey of nearby states, and find that one of our methods, ${\rho}$-explore, consistently outperforms the Double DQN baseline in an discrete environment by 49\% in terms of Eval Reward Return.

翻译：强化学习是建模决策过程的强大工具，但其依赖的探索-利用权衡仍是许多任务面临的开放挑战。本文基于"对于早期智能体而言，考虑从邻近状态约束区域推导的动作可能产生更优探索行为"这一直觉，研究了基于邻域状态的无模型探索方法。我们提出了两种基于邻近状态调研来选择探索动作的算法，并发现其中一种方法——$\rho$-探索（${\rho}$-explore）——在离散环境中以评估奖励回报（Eval Reward Return）为指标，持续比Double DQN基线方法提升49%的性能。