This paper focuses on the information freshness of finite-state Markov sources, using the uncertainty of information (UoI) as the performance metric. Measured by Shannon's entropy, UoI can capture not only the transition dynamics of the Markov source but also the different evolutions of information quality caused by the different values of the last observation. We consider an information update system with M finite-state Markov sources transmitting information to a remote monitor via m communication channels. Our goal is to explore the optimal scheduling policy to minimize the sum-UoI of the Markov sources. The problem is formulated as a restless multi-armed bandit (RMAB). We relax the RMAB and then decouple the relaxed problem into M single bandit problems. Analyzing the single bandit problem provides useful properties with which the relaxed problem reduces to maximizing a concave and piecewise linear function, allowing us to develop a gradient method to solve the relaxed problem and obtain its optimal policy. By rounding up the optimal policy for the relaxed problem, we obtain an index policy for the original RMAB problem. Notably, the proposed index policy is universal in the sense that it applies to general RMABs with bounded cost functions.

翻译：本文聚焦于有限状态马尔可夫源的信息新鲜度，采用信息不确定性（UoI）作为性能度量指标。以香农熵度量的UoI不仅能捕捉马尔可夫源的转移动态特性，还能反映由上次观测值差异导致的信息质量的不同演化过程。我们研究了一个信息更新系统，该系统包含M个有限状态马尔可夫源，通过m条通信信道向远程监控器传输信息。目标是探索最优调度策略以最小化马尔可夫源的UoI总和。该问题被建模为无休止多臂赌博机（RMAB）。我们松弛了RMAB约束，并将松弛问题解耦为M个单赌博机问题。通过分析单赌博机问题，我们获得了有用的性质，使得松弛问题简化为最大化一个凹分段线性函数，进而开发出梯度方法求解松弛问题并获得其最优策略。通过对松弛问题的最优策略进行舍入，我们得到了原始RMAB问题的索引策略。值得注意的是，所提出的索引策略具有普适性，可适用于具有有界成本函数的一般RMAB问题。