Path-planning for autonomous vehicles in threat-laden environments is a fundamental challenge because the problem is nonlinear and nonconvex even in simplest scenarios. While traditional optimal control methods can be used to find ideal paths, the computational time is often too slow for real-time decision-making. To solve this challenge, we propose a method based on Deep Deterministic Policy Gradient (DDPG) and model the threat as possibly multiple circular 'no-go' zones. A mission is regarded as a failure if the vehicle enters this restricted zone at any time or does not reach a neighborhood of the destination. The DDPG agent is trained through trial and error in a simulated environment, learning a direct mapping from its current state (position and heading) to a series of feasible actions that guide the agent to safely reach its destination. The reword function has three parts: (a) an attractive field centered at the final destination, (b) some repulsive fields centered at the origins of circular obstacles, and (c) a penalty of control energy consumption (the magnitude of heading change) that indirectly in favor for straight path. The DDPG trains the agent using these incentives to find the largest possible set of starting points wherein a safe path to the destination is guaranteed. This provides critical information for mission planning, showing beforehand whether a task is achievable from a given starting point, assisting pre-mission planning activities. The approach is validated in simulation. A comparison between the DDPG method and a traditional optimal control (pseudo-spectral) method is carried out. The results show that the learning-based agent produces effective paths while being significantly faster, making it a better fit for real-time applications.

翻译：在威胁环境下自主车辆的路径规划是一项基本挑战，因为即便在最简单的场景中，该问题也是非线性和非凸的。虽然传统最优控制方法可用于寻找理想路径，但其计算时间往往过慢，无法满足实时决策需求。为解决这一挑战，我们提出了一种基于深度确定性策略梯度（DDPG）的方法，并将威胁建模为可能的多个圆形"禁行"区域。若车辆在任何时刻进入该受限区域或未到达目的地邻域，则任务被视为失败。DDPG智能体通过模拟环境中的试错训练，学习从当前状态（位置和航向）到一系列可行动作的直接映射，引导智能体安全到达目的地。奖励函数包含三部分：(a) 以最终目的地为中心的引力场，(b) 以圆形障碍物为中心的一些斥力场，(c) 控制能量消耗（航向变化幅度）的惩罚项，该惩罚间接有利于直线路径。DDPG利用这些激励训练智能体，从而找到尽可能大的起始点集合，确保从这些起始点到目的地存在安全路径。这为任务规划提供了关键信息，能预先判断给定起始点是否可实现任务，辅助任务前规划活动。该方法在仿真中得到了验证。我们将DDPG方法与传统最优控制（伪谱）方法进行了对比。结果表明，基于学习的智能体在生成有效路径的同时，速度显著更快，因此更适合实时应用场景。