Exploiting unmanned aerial vehicles (UAVs) to execute tasks is gaining growing popularity recently. To solve the underlying task scheduling problem, the deep reinforcement learning (DRL) based methods demonstrate notable advantage over the conventional heuristics as they rely less on hand-engineered rules. However, their decision space will become prohibitively huge as the problem scales up, thus deteriorating the computation efficiency. To alleviate this issue, we propose a double-level deep reinforcement learning (DL-DRL) approach based on a divide and conquer framework (DCF), where we decompose the task scheduling of multi-UAV into task allocation and route planning. Particularly, we design an encoder-decoder structured policy network in our upper-level DRL model to allocate the tasks to different UAVs, and we exploit another attention based policy network in our lower-level DRL model to construct the route for each UAV, with the objective to maximize the number of executed tasks given the maximum flight distance of the UAV. To effectively train the two models, we design an interactive training strategy (ITS), which includes pre-training, intensive training and alternate training. Experimental results show that our DL-DRL performs favorably against the learning-based and conventional baselines including the OR-Tools, in terms of solution quality and computation efficiency. We also verify the generalization performance of our approach by applying it to larger sizes of up to 1000 tasks. Moreover, we also show via an ablation study that our ITS can help achieve a balance between the performance and training efficiency.

翻译：利用无人机执行任务近年来越发受到关注。为解决底层任务调度问题，基于深度强化学习的方法相较于传统启发式方法具有显著优势，因其较少依赖人工设计的规则。然而，随着问题规模扩大，其决策空间将急剧膨胀，导致计算效率恶化。为缓解该问题，本文提出一种基于分治框架的双层深度强化学习方法，将多无人机任务调度分解为任务分配与路径规划两个子问题。具体而言，我们在上层深度强化学习模型中设计了一个编码器-解码器结构的策略网络，用于将任务分配给不同无人机；同时在下层深度强化学习模型中利用另一个基于注意力机制的策略网络，为每架无人机构建飞行路径，其目标是在给定最大飞行距离约束下最大化可执行任务数量。为有效训练这两个模型，我们设计了一种交互式训练策略，包含预训练、强化训练和交替训练三个阶段。实验结果表明，我们的DL-DRL方法在解质量和计算效率方面均优于包括OR-Tools在内的基于学习的传统基线方法。我们还将该方法扩展应用于多达1000个任务的大规模场景，验证了其泛化性能。此外，消融实验表明，所提出的交互式训练策略有助于在性能与训练效率之间取得平衡。