Unmanned aerial vehicles (UAVs) are capable of surveying expansive areas, but their operational range is constrained by limited battery capacity. The deployment of mobile recharging stations using unmanned ground vehicles (UGVs) significantly extends the endurance and effectiveness of UAVs. However, optimizing the routes of both UAVs and UGVs, known as the UAV-UGV cooperative routing problem, poses substantial challenges, particularly with respect to the selection of recharging locations. Here in this paper, we leverage reinforcement learning (RL) for the purpose of identifying optimal recharging locations while employing constraint programming to determine cooperative routes for the UAV and UGV. Our proposed framework is then benchmarked against a baseline solution that employs Genetic Algorithms (GA) to select rendezvous points. Our findings reveal that RL surpasses GA in terms of reducing overall mission time, minimizing UAV-UGV idle time, and mitigating energy consumption for both the UAV and UGV. These results underscore the efficacy of incorporating heuristics to assist RL, a method we refer to as heuristics-assisted RL, in generating high-quality solutions for intricate routing problems.

翻译：摘要：无人机能够勘测广阔区域，但其作业范围受限于有限的电池容量。通过地面无人车部署移动充电站显著延长了无人机的续航能力和效能。然而，优化无人机与地面无人车的路径（即无人机-地面无人车协同路径规划问题）带来了重大挑战，尤其是在充电位置的选择方面。本文利用强化学习来识别最优充电位置，同时采用约束规划确定无人机与地面无人车的协同路径。随后，我们以使用遗传算法选择汇合点的基线方案为基准，对所提出的框架进行了评估。研究结果表明，强化学习在减少整体任务时间、最小化无人机-地面无人车空闲时间以及降低两者能耗方面均优于遗传算法。这些结果凸显了引入启发式方法辅助强化学习（我们称之为启发式辅助强化学习）在生成复杂路径规划问题高质量解方面的有效性。