Autonomous robots exploring unknown environments face a significant challenge: navigating effectively without prior maps and with limited external feedback. This challenge intensifies in sparse reward environments, where traditional exploration techniques often fail. In this paper, we present TopoNav, a novel topological navigation framework that integrates active mapping, hierarchical reinforcement learning, and intrinsic motivation to enable efficient goal-oriented exploration and navigation in sparse-reward settings. TopoNav dynamically constructs a topological map of the environment, capturing key locations and pathways. A two-level hierarchical policy architecture, comprising a high-level graph traversal policy and low-level motion control policies, enables effective navigation and obstacle avoidance while maintaining focus on the overall goal. Additionally, TopoNav incorporates intrinsic motivation to guide exploration toward relevant regions and frontier nodes in the topological map, addressing the challenges of sparse extrinsic rewards. We evaluate TopoNav both in the simulated and real-world off-road environments using a Clearpath Jackal robot, across three challenging navigation scenarios: goal-reaching, feature-based navigation, and navigation in complex terrains. We observe an increase in exploration coverage by 7- 20%, in success rates by 9-19%, and reductions in navigation times by 15-36% across various scenarios, compared to state-of-the-art methods

翻译：自主机器人在探索未知环境时面临一个重大挑战：在没有先验地图且外部反馈有限的情况下进行有效导航。这一挑战在稀疏奖励环境中尤为突出，传统探索技术在此类环境中往往失效。本文提出TopoNav——一种新颖的拓扑导航框架，该框架融合主动建图、分层强化学习与内在动机机制，能够在稀疏奖励场景中实现高效的目标导向探索与导航。TopoNav动态构建环境的拓扑地图，捕捉关键位置与路径通道。采用包含高层图遍历策略与底层运动控制策略的双层分层策略架构，在保持对总体目标关注的同时实现有效导航与避障。此外，TopoNav引入内在动机机制，引导探索朝向拓扑地图中的相关区域与边界节点，以应对稀疏外部奖励带来的挑战。我们在仿真与真实野外环境中使用Clearpath Jackal机器人，通过目标抵达、基于特征的导航及复杂地形导航三种挑战性场景对TopoNav进行评估。实验表明，相较于前沿方法，TopoNav在不同场景中实现了7-20%的探索覆盖率提升、9-19%的成功率提升以及15-36%的导航时间缩减。