Unmanned aerial vehicles (UAVs) have the potential to greatly aid Internet of Things (IoT) networks in mission-critical data collection, thanks to their flexibility and cost-effectiveness. However, challenges arise due to the UAV's limited onboard energy and the unpredictable status updates from sensor nodes (SNs), which impact the freshness of collected data. In this paper, we investigate the energy-efficient and timely data collection in IoT networks through the use of a solar-powered UAV. Each SN generates status updates at stochastic intervals, while the UAV collects and subsequently transmits these status updates to a central data center. Furthermore, the UAV harnesses solar energy from the environment to maintain its energy level above a predetermined threshold. To minimize both the average age of information (AoI) for SNs and the energy consumption of the UAV, we jointly optimize the UAV trajectory, SN scheduling, and offloading strategy. Then, we formulate this problem as a Markov decision process (MDP) and propose a meta-reinforcement learning algorithm to enhance the generalization capability. Specifically, the compound-action deep reinforcement learning (CADRL) algorithm is proposed to handle the discrete decisions related to SN scheduling and the UAV's offloading policy, as well as the continuous control of UAV flight. Moreover, we incorporate meta-learning into CADRL to improve the adaptability of the learned policy to new tasks. To validate the effectiveness of our proposed algorithms, we conduct extensive simulations and demonstrate their superiority over other baseline algorithms.

翻译：无人机（UAV）凭借其灵活性和成本效益，有望在关键任务数据收集中为物联网（IoT）网络提供重要支持。然而，由于无人机机载能量有限以及传感器节点（SN）状态更新的不可预测性，所收集数据的时效性面临挑战。本文研究利用太阳能无人机实现物联网网络中能效优先的及时数据收集。每个传感器节点以随机间隔生成状态更新信息，无人机收集这些信息后将其传输至中央数据中心。此外，无人机通过从环境中采集太阳能，将自身能量水平维持在预设阈值以上。为最小化传感器节点的平均信息年龄（AoI）和无人机能量消耗，我们联合优化了无人机轨迹、传感器节点调度及卸载策略。随后将该问题建模为马尔可夫决策过程（MDP），并提出一种元强化学习算法以增强泛化能力。具体而言，我们提出复合动作深度强化学习（CADRL）算法，该算法可处理传感器节点调度与无人机卸载策略的离散决策，以及无人机飞行的连续控制问题。此外，我们将元学习融入CADRL，以提升所学策略对新任务的适应性。为验证所提算法的有效性，我们进行了广泛仿真实验，并证明了其相较于其他基线算法的优越性。