In this paper, we explore how to schedule multiple users to optimize information freshness in a pull-based wireless network, where the status updates from users are requested by randomly arriving queries at the destination. We use the age of information at query (QAoI) to characterize the performance of information freshness. Such a decision-making problem is naturally modeled as a Markov decision process (MDP), which, however, is prohibitively high to be solved optimally by the standard method due to the curse of dimensionality. To address this issue, we employ Whittle index approach, which allows us to decouple the original MDP into multiple sub-MDPs by relaxing the scheduling constraints. However, the binary Markovian query arrival process results in a bi-dimensional state and complex state transitions within each sub-MDP, making it challenging to verify Whittle indexability using conventional methods. After a thorough analysis of the sub-MDP's structure, we show that it is unichain and its optimal policy follows a threshold-type structure. This facilitates the verification of Whittle indexability of the sub-MDP by employing an easy-to-verify condition. Subsequently, the steady-state probability distributions of the sub-MDP under different threshold-type policies are analyzed, constituting the analytical expressions of different Whittle indices in terms of the expected average QAoI and scheduling time of the sub-MDP. Building on these, we devise an efficient algorithm to calculate Whittle indices for the formulated sub-MDPs. The simulation results validate our analyses and show the proposed Whittle index policy outperforms baseline policies and achieves near-optimal performance.

翻译：本文研究如何在基于拉取的无线网络中调度多个用户以优化信息新鲜度，其中目的端通过随机到达的查询请求用户的状态更新。我们使用查询时刻信息年龄（QAoI）来刻画信息新鲜度的性能。该决策问题自然建模为马尔可夫决策过程（MDP），但由于维数灾难，采用标准方法求解最优解的计算复杂度极高。为解决此问题，我们采用Whittle指数方法，通过松弛调度约束将原始MDP解耦为多个子MDP。然而，二值马尔可夫查询到达过程导致每个子MDP具有二维状态和复杂的状态转移，使得利用传统方法验证Whittle可索引性面临挑战。通过对子MDP结构的深入分析，我们证明其具有单链性质且最优策略呈阈值型结构。这为利用易于验证的条件来检验子MDP的Whittle可索引性提供了便利。随后，我们分析了子MDP在不同阈值型策略下的稳态概率分布，从而构建了基于子MDP期望平均QAoI与调度时间的Whittle指数解析表达式。在此基础上，我们设计了一种高效算法来计算所构建子MDP的Whittle指数。仿真结果验证了我们的分析，并表明所提出的Whittle指数策略优于基线策略，且能实现接近最优的性能。