Integrated Sensing and Communication (ISAC) enables joint data transmission and environmental perception for sixth-generation (6G) networks, but centralized and virtualized RAN control loops introduce telemetry latency that yields stale observations and unstable control. This paper proposes a Digital Twin-assisted belief-state reinforcement learning framework for latency-robust ISAC. A Digital Twin (DT) reconstructs a synchronized belief state from delayed telemetry using an Extended Kalman Filter, and a Proximal Policy Optimization agent performs joint beamforming and power allocation for communication and sensing. Closed-loop simulations with telemetry delays up to 100 ms demonstrate consistent performance gains over latency-unaware deep reinforcement learning (DRL) and heuristic baselines. At 50 ms latency, the proposed method improves median throughput by 12% and reduces sensing error by 7% relative to a DT-only controller, while achieving an order-of-magnitude reduction in reliability violations. Even at 100 ms latency, the proposed approach retains approximately 88% of its zero-latency throughput. These results show that Digital Twin-assisted belief-state control enables stable and efficient ISAC operation under realistic telemetry delays in 6G networks.

翻译：集成感知与通信（Integrated Sensing and Communication, ISAC）可实现第六代（6G）网络的联合数据传输与环境感知，但集中式与虚拟化RAN控制环引入了遥测延迟，导致状态观测陈旧及控制不稳定。本文提出一种数字孪生辅助的信念状态强化学习框架，用于实现延迟鲁棒的ISAC。该框架中，数字孪生（Digital Twin, DT）利用扩展卡尔曼滤波器从延迟遥测数据中重构同步信念状态，并由近端策略优化智能体执行通信与感知的联合波束赋形与功率分配。在遥测延迟高达100毫秒的闭环仿真中，所提方法相比延迟无关的深度强化学习（Deep Reinforcement Learning, DRL）及启发式基线方法持续展现出性能优势。在50毫秒延迟下，相比仅采用DT的控制器，所提方法将中位吞吐量提升12%，感知误差降低7%，同时将可靠性违规指标降低一个数量级。即便在100毫秒延迟下，所提方法仍可保留约88%的零延迟吞吐量。结果表明，数字孪生辅助的信念状态控制能在6G网络实际遥测延迟下实现稳定高效的ISAC运行。