Optimizing edge caching is crucial for the advancement of next-generation (nextG) wireless networks, ensuring high-speed and low-latency services for mobile users. Existing data-driven optimization approaches often lack awareness of the distribution of random data variables and focus solely on optimizing cache hit rates, neglecting potential reliability concerns, such as base station overload and unbalanced cache issues. This oversight can result in system crashes and degraded user experience. To bridge this gap, we introduce a novel digital twin-assisted optimization framework, called D-REC, which integrates reinforcement learning (RL) with diverse intervention modules to ensure reliable caching in nextG wireless networks. We first develop a joint vertical and horizontal twinning approach to efficiently create network digital twins, which are then employed by D-REC as RL optimizers and safeguards, providing ample datasets for training and predictive evaluation of our cache replacement policy. By incorporating reliability modules into a constrained Markov decision process, D-REC can adaptively adjust actions, rewards, and states to comply with advantageous constraints, minimizing the risk of network failures. Theoretical analysis demonstrates comparable convergence rates between D-REC and vanilla data-driven methods without compromising caching performance. Extensive experiments validate that D-REC outperforms conventional approaches in cache hit rate and load balancing while effectively enforcing predetermined reliability intervention modules.

翻译：优化边缘缓存对于下一代无线网络的发展至关重要，可确保为移动用户提供高速低时延服务。现有的数据驱动优化方法通常缺乏对随机数据变量分布的感知，且仅专注于优化缓存命中率，忽视了潜在的可靠性问题，例如基站过载与缓存不均衡。这种疏忽可能导致系统崩溃与用户体验下降。为弥补这一不足，我们提出了一种新颖的数字孪生辅助优化框架，称为D-REC，该框架将强化学习与多种干预模块相结合，以确保下一代无线网络中的可靠缓存。我们首先开发了一种联合垂直与水平孪生方法，以高效构建网络数字孪生体，随后D-REC将其用作强化学习优化器与保障机制，为我们的缓存替换策略提供充足的训练数据集与预测评估。通过将可靠性模块整合到约束马尔可夫决策过程中，D-REC能够自适应地调整动作、奖励与状态，以遵循有利约束，从而最小化网络故障风险。理论分析表明，D-REC与原始数据驱动方法在收敛速度上相当，且未牺牲缓存性能。大量实验验证了D-REC在缓存命中率与负载均衡方面优于传统方法，同时能有效执行预定的可靠性干预模块。