Unmanned Aerial Vehicles (UAVs) are increasingly populating urban areas for delivery and surveillance purposes. In this work, we develop an optimal navigation strategy based on Deep Reinforcement Learning. The environment is represented by a three-dimensional high-fidelity simulation of an urban flow, characterized by turbulence and recirculation zones. The algorithm presented here is a flow-aware Proximal Policy Optimization (PPO) combined with a Gated Transformer eXtra Large (GTrXL) architecture, giving the agent richer information about the turbulent flow field in which it navigates. The results are compared with a PPO+GTrXL without the secondary prediction tasks, a PPO combined with Long Short Term Memory (LSTM) cells and a traditional navigation algorithm. The obtained results show a significant increase in the success rate (SR) and a lower crash rate (CR) compared to a PPO+LSTM, PPO+GTrXL and the classical Zermelo's navigation algorithm, paving the way to a completely reimagined UAV landscape in complex urban environments.

翻译：无人机（UAV）在城区的配送与监视任务中日益普及。本研究提出了一种基于深度强化学习的最优导航策略。环境采用高保真三维城市流场模拟，其特征包括湍流与回流区。本文算法为一种流场感知的近端策略优化（PPO）方法，结合门控Transformer超大规模（GTrXL）架构，使智能体在导航过程中能获取更丰富的湍流场信息。研究结果与无辅助预测任务的PPO+GTrXL、结合长短期记忆（LSTM）单元的PPO以及传统导航算法进行了对比。实验表明，相较于PPO+LSTM、PPO+GTrXL及经典的Zermelo导航算法，所提方法在成功率（SR）上显著提升，碰撞率（CR）降低，为复杂城市环境下无人机应用范式的革新开辟了道路。