Multi-rotor UAVs suffer from a restricted range and flight duration due to limited battery capacity. Autonomous landing on a 2D moving platform offers the possibility to replenish batteries and offload data, thus increasing the utility of the vehicle. Classical approaches rely on accurate, complex and difficult-to-derive models of the vehicle and the environment. Reinforcement learning (RL) provides an attractive alternative due to its ability to learn a suitable control policy exclusively from data during a training procedure. However, current methods require several hours to train, have limited success rates and depend on hyperparameters that need to be tuned by trial-and-error. We address all these issues in this work. First, we decompose the landing procedure into a sequence of simpler, but similar learning tasks. This is enabled by applying two instances of the same RL based controller trained for 1D motion for controlling the multi-rotor's movement in both the longitudinal and the lateral directions. Second, we introduce a powerful state space discretization technique that is based on i) kinematic modeling of the moving platform to derive information about the state space topology and ii) structuring the training as a sequential curriculum using transfer learning. Third, we leverage the kinematics model of the moving platform to also derive interpretable hyperparameters for the training process that ensure sufficient maneuverability of the multi-rotor vehicle. The training is performed using the tabular RL method Double Q-Learning. Through extensive simulations we show that the presented method significantly increases the rate of successful landings, while requiring less training time compared to other deep RL approaches. Finally, we deploy and demonstrate our algorithm on real hardware. For all evaluation scenarios we provide statistics on the agent's performance.

翻译：多旋翼无人机因电池容量有限，其航程和飞行时长受到限制。在二维移动平台上自主降落为电池补给和数据卸载提供了可行性，从而提升了无人机的实用性。传统方法依赖于对飞行器和环境建立精确、复杂且难以推导的模型。强化学习（RL）因其能在训练过程中仅从数据中学习合适的控制策略，成为一种颇具吸引力的替代方案。然而，现有方法需要数小时训练，成功率有限，且依赖需通过试错调整的超参数。本研究针对上述问题展开：首先，将降落过程分解为一系列更简单但相似的子任务。通过应用同一基于RL的控制器的两个实例（分别针对纵向和横向一维运动进行训练）来实现对多旋翼无人机这两个方向运动的控制。其次，我们提出了一种高效的状态空间离散化技术，该技术基于：i）对移动平台进行运动学建模，以推导状态空间拓扑结构信息；ii）利用迁移学习将训练过程构建为顺序课程学习。第三，我们利用移动平台的运动学模型，推导出训练过程中可解释的超参数，以确保多旋翼飞行器具备足够的机动能力。训练采用表格化RL方法Double Q-Learning。通过大量仿真对比，所提方法相较其他深度RL方法显著提升了着陆成功率，同时缩短了训练时间。最后，我们在真实硬件上部署并验证了该算法。所有评估场景均提供了智能体性能的统计数据。