Data freshness, measured by Age of Information (AoI), is highly relevant in networked applications such as Vehicle to Everything (V2X), smart health systems, and Industrial Internet of Things (IIoT). However, freshness alone does not always equate to utility in decision-making. In decision-critical settings, some stale data may be more valuable than fresh updates. Motivated by this, we move beyond AoI-centric policies and investigate how data staleness affects remote decision-making effectiveness under random delay and limited communication resources. To this end, we propose AR-MDP, an Age-aware Remote Markov Decision Process framework, which co-designs optimal sampling and remote decision-making under a sampling frequency constraint and random delay. To efficiently solve this problem, we design a new two-stage hierarchical algorithm, namely Quick Bellman-Linear-Program (QuickBLP), where the first stage involves solving the Dinkelbach root of a Bellman variant and the second stage involves solving a streamlined linear program (LP). For the tricky first stage, we propose a new One-layer Primal-Dinkelbach Synchronous Iteration (OnePDSI) method, which overcomes the re-convergence and non-expansive divergence present in existing per-sample multi-layer algorithms. Through rigorous convergence analysis of our proposed algorithms, we establish that the worst-case optimality gap in OnePDSI exhibits exponential decay with respect to iteration $K$ at a rate of $\mathcal{O}(\frac{1}{R^K})$. Through sensitivity analysis, we derive a threshold for the sampling frequency, beyond which additional sampling does not yield further gains in decision-making. Simulation results validate our analyses.

翻译：数据新鲜度，以信息年龄（AoI）衡量，在车联网（V2X）、智能健康系统和工业物联网（IIoT）等网络应用中至关重要。然而，在决策过程中，新鲜度本身并不总能等同于效用。在决策关键场景中，某些过时数据可能比新鲜更新更有价值。受此启发，我们超越了以AoI为中心的策略，研究了在随机延迟和有限通信资源下，数据陈旧性如何影响远程决策的有效性。为此，我们提出了AR-MDP，一个年龄感知远程马尔可夫决策过程框架，该框架在采样频率约束和随机延迟下，协同设计了最优采样与远程决策。为高效求解此问题，我们设计了一种新的两阶段分层算法，即快速贝尔曼-线性规划（QuickBLP），其中第一阶段涉及求解一个贝尔曼变体的Dinkelbach根，第二阶段涉及求解一个简化的线性规划（LP）。针对棘手的第一阶段，我们提出了一种新的单层原始-Dinkelbach同步迭代（OnePDSI）方法，该方法克服了现有逐样本多层算法中存在的重新收敛和非扩张发散问题。通过对所提算法进行严格的收敛性分析，我们证明了OnePDSI的最坏情况最优性间隙相对于迭代次数$K$以$\mathcal{O}(\frac{1}{R^K})$的速率呈指数衰减。通过敏感性分析，我们推导出一个采样频率阈值，超过该阈值后，额外的采样不会带来决策效果的进一步提升。仿真结果验证了我们的分析。