Unmanned aerial vehicles (UAVs) are increasingly deployed to provide wireless connectivity to static and mobile ground users in situations of increased network demand or points of failure in existing terrestrial cellular infrastructure. However, UAVs are energy-constrained and experience the challenge of interference from nearby UAV cells sharing the same frequency spectrum, thereby impacting the system's energy efficiency (EE). Recent approaches focus on optimising the system's EE by optimising the trajectory of UAVs serving only static ground users and neglecting mobile users. Several others neglect the impact of interference from nearby UAV cells, assuming an interference-free network environment. Despite growing research interest in decentralised control over centralised UAVs' control, direct collaboration among UAVs to improve coordination while optimising the systems' EE has not been adequately explored. To address this, we propose a direct collaborative communication-enabled multi-agent decentralised double deep Q-network (CMAD-DDQN) approach. The CMAD-DDQN is a collaborative algorithm that allows UAVs to explicitly share their telemetry via existing 3GPP guidelines by communicating with their nearest neighbours. This allows the agent-controlled UAVs to optimise their 3D flight trajectories by filling up knowledge gaps and converging to optimal policies. Simulation results show that the proposed approach outperforms existing baselines in terms of maximising the systems' EE without degrading coverage performance in the network. The CMAD-DDQN approach outperforms the MAD-DDQN that neglects direct collaboration among UAVs, the multi-agent deep deterministic policy gradient (MADDPG) and random policy approaches that consider a 2D UAV deployment design while neglecting interference from nearby UAV cells by about 15%, 65% and 85%, respectively.

翻译：无人机正越来越多地部署于网络需求激增或现有地面蜂窝基础设施故障场景，为静态与移动地面用户提供无线连接。然而，无人机受限于能量约束，且面临共享同一频谱的邻近无人机小区的干扰挑战，从而影响系统能效。现有研究多聚焦于通过优化仅服务静态地面用户的无人机轨迹来提升系统能效，忽略了移动用户需求；另有部分研究假设无干扰网络环境，忽视邻近无人机小区干扰的影响。尽管分散式无人机控制相较于集中式控制的研究兴趣日益增长，但通过无人机间直接协作来改善协调性以优化系统能效的问题尚未得到充分探索。为此，本文提出一种基于直接协作通信的多智能体分散式双深度Q网络方法。该协作算法允许无人机依据现有3GPP指南，通过与最近邻节点通信显式共享遥测数据，使智能体控制的无人机能够填补知识空缺并收敛至最优策略，从而优化其三维飞行轨迹。仿真结果表明，所提方法在不降低网络覆盖性能的前提下，在最大化系统能效方面优于现有基准方案。与忽略无人机间直接协作的多智能体分散式双深度Q网络、采用二维无人机部署设计但忽视邻近无人机小区干扰的多智能体深度确定性策略梯度及随机策略方法相比，所提方法分别实现了约15%、65%和85%的性能提升。