In this paper, we explore how to schedule multiple users to optimize information freshness in a pull-based wireless network, where the status updates from users are requested by randomly arriving queries at the destination. We use the age of information at query (QAoI) to characterize the performance of information freshness. Such a decision-making problem is naturally modeled as a Markov decision process (MDP), which, however, is prohibitively high to be solved optimally by the standard method due to the curse of dimensionality. To address this issue, we employ Whittle index approach, which allows us to decouple the original MDP into multiple sub-MDPs by relaxing the scheduling constraints. However, the binary Markovian query arrival process results in a bi-dimensional state and complex state transitions within each sub-MDP, making it challenging to verify Whittle indexability using conventional methods. After a thorough analysis of the sub-MDP's structure, we show that it is unichain and its optimal policy follows a threshold-type structure. This facilitates the verification of Whittle indexability of the sub-MDP by employing an easy-to-verify condition. Subsequently, the steady-state probability distributions of the sub-MDP under different threshold-type policies are analyzed, constituting the analytical expressions of different Whittle indices in terms of the expected average QAoI and scheduling time of the sub-MDP. Building on these, we devise an efficient algorithm to calculate Whittle indices for the formulated sub-MDPs. The simulation results validate our analyses and show the proposed Whittle index policy outperforms baseline policies and achieves near-optimal performance.

翻译：本文研究在基于查询的无线网络中如何调度多个用户以优化信息新鲜度，其中目的节点通过随机到达的查询请求获取用户的状态更新。我们采用查询时刻信息年龄（QAoI）来刻画信息新鲜度的性能指标。该决策问题自然建模为马尔可夫决策过程（MDP），然而由于维度灾难问题，采用标准方法求解最优策略的计算复杂度极高。为解决此问题，我们采用Whittle索引方法，通过松弛调度约束将原始MDP解耦为多个子MDP。然而，二元马尔可夫查询到达过程导致每个子MDP具有二维状态空间和复杂的状态转移，使得传统方法难以验证Whittle可索引性。通过对子MDP结构的深入分析，我们证明其具有单链特性且最优策略呈现阈值型结构。这一特性使得我们可以通过易于验证的条件来判定子MDP的Whittle可索引性。随后，我们分析了子MDP在不同阈值策略下的稳态概率分布，进而推导出基于子MDP期望平均QAoI与调度时间的Whittle索引解析表达式。在此基础上，我们设计了一种高效算法来计算所构建子MDP的Whittle索引。仿真结果验证了理论分析的有效性，并表明所提出的Whittle索引策略在性能上优于基准策略，且能达到接近最优的性能水平。