Unmanned aerial vehicles (UAVs) serving as aerial base stations can be deployed to provide wireless connectivity to mobile users, such as vehicles. However, the density of vehicles on roads often varies spatially and temporally primarily due to mobility and traffic situations in a geographical area, making it difficult to provide ubiquitous service. Moreover, as energy-constrained UAVs hover in the sky while serving mobile users, they may be faced with interference from nearby UAV cells or other access points sharing the same frequency band, thereby impacting the system's energy efficiency (EE). Recent multi-agent reinforcement learning (MARL) approaches applied to optimise the users' coverage worked well in reasonably even densities but might not perform as well in uneven users' distribution, i.e., in urban road networks with uneven concentration of vehicles. In this work, we propose a density-aware communication-enabled multi-agent decentralised double deep Q-network (DACEMAD-DDQN) approach that maximises the total system's EE by jointly optimising the trajectory of each UAV, the number of connected users, and the UAVs' energy consumption while keeping track of dense and uneven users' distribution. Our result outperforms state-of-the-art MARL approaches in terms of EE by as much as 65% - 85%.

翻译：无人机作为空中基站，可用于为车辆等移动用户提供无线连接。然而，由于地理区域内车辆移动性和交通状况的时空差异，道路上的车辆密度常呈现动态变化，使得提供广域服务面临挑战。此外，能量受限的无人机在为移动用户服务时悬停空中，可能受到邻近无人机小区或共享同一频段接入点的干扰，从而影响系统能效。现有应用于优化用户覆盖的多智能体强化学习方法在用户密度均匀场景下表现良好，但在用户分布不均（如城市道路网络中车辆分布不均）时效果欠佳。本文提出一种密度感知通信增强的多智能体分散双深度Q网络方法，通过联合优化每架无人机的飞行轨迹、连接用户数量及能耗，同时追踪密集且不均匀的用户分布，最大化系统总能效。实验结果表明，本方法在能效指标上较当前最优的多智能体强化学习方法提升达65%-85%。