Equipping drones with target search capabilities is highly desirable for applications in disaster rescue and smart warehouse delivery systems. Multiple intelligent drones that can collaborate with each other and maneuver among obstacles show more effectiveness in accomplishing tasks in a shorter amount of time. However, carrying out collaborative target search (CTS) without prior target information is extremely challenging, especially with a visual drone swarm. In this work, we propose a novel data-efficient deep reinforcement learning (DRL) approach called adaptive curriculum embedded multistage learning (ACEMSL) to address these challenges, mainly 3-D sparse reward space exploration with limited visual perception and collaborative behavior requirements. Specifically, we decompose the CTS task into several subtasks including individual obstacle avoidance, target search, and inter-agent collaboration, and progressively train the agents with multistage learning. Meanwhile, an adaptive embedded curriculum (AEC) is designed, where the task difficulty level (TDL) can be adaptively adjusted based on the success rate (SR) achieved in training. ACEMSL allows data-efficient training and individual-team reward allocation for the visual drone swarm. Furthermore, we deploy the trained model over a real visual drone swarm and perform CTS operations without fine-tuning. Extensive simulations and real-world flight tests validate the effectiveness and generalizability of ACEMSL. The project is available at https://github.com/NTU-UAVG/CTS-visual-drone-swarm.git.

翻译：为无人机装备目标搜索能力在灾难救援与智能仓储物流系统中具有重要应用价值。多架可相互协同并在障碍物间机动的智能无人机，能够在更短时间内高效完成任务。然而，在无先验目标信息条件下开展协同目标搜索（CTS）极具挑战性，特别是对视觉无人机群而言。本文提出一种名为自适应嵌入式课程多阶段学习（ACEMSL）的新型数据高效深度强化学习（DRL）方法，以应对上述挑战，主要解决有限视觉感知下的三维稀疏奖励空间探索及协同行为需求问题。具体而言，我们将CTS任务分解为个体避障、目标搜索和智能体间协作等子任务，并通过多阶段学习渐进式训练智能体。同时，我们设计了自适应嵌入式课程（AEC），使任务难度等级（TDL）可根据训练过程中实现的成功率（SR）自适应调整。ACEMSL能够实现视觉无人机群的数据高效训练及个体-团队奖励分配。此外，我们将训练好的模型直接部署至真实视觉无人机群，无需微调即可执行CTS操作。大量仿真实验与实飞测试验证了ACEMSL的有效性与泛化能力。项目代码已在GitHub开源：https://github.com/NTU-UAVG/CTS-visual-drone-swarm.git。