Millimeter-wave (mmWave) communication is a vital component of future generations of mobile networks, offering not only high data rates but also precise beams, making it ideal for indoor navigation in complex environments. However, the challenges of multipath propagation and noisy signal measurements in indoor spaces complicate the use of mmWave signals for navigation tasks. Traditional physics-based methods, such as following the angle of arrival (AoA), often fall short in complex scenarios, highlighting the need for more sophisticated approaches. Digital twins, as virtual replicas of physical environments, offer a powerful tool for simulating and optimizing mmWave signal propagation in such settings. By creating detailed, physics-based models of real-world spaces, digital twins enable the training of machine learning algorithms in virtual environments, reducing the costs and limitations of physical testing. Despite their advantages, current machine learning models trained in digital twins often overfit specific virtual environments and require costly retraining when applied to new scenarios. In this paper, we propose a Physics-Informed Reinforcement Learning (PIRL) approach that leverages the physical insights provided by digital twins to shape the reinforcement learning (RL) reward function. By integrating physics-based metrics such as signal strength, AoA, and path reflections into the learning process, PIRL enables efficient learning and improved generalization to new environments without retraining. Our experiments demonstrate that the proposed PIRL, supported by digital twin simulations, outperforms traditional heuristics and standard RL models, achieving zero-shot generalization in unseen environments and offering a cost-effective, scalable solution for wireless indoor navigation.

翻译：毫米波通信是未来移动网络的关键组成部分，它不仅提供高数据速率，还能产生精确波束，因而成为复杂环境下室内导航的理想选择。然而，室内空间中的多径传播和噪声信号测量等挑战，使得利用毫米波信号进行导航任务变得复杂。传统的基于物理的方法（如跟踪到达角）在复杂场景中往往效果不佳，这凸显了对更先进方法的需求。数字孪生作为物理环境的虚拟副本，为模拟和优化此类环境中的毫米波信号传播提供了强大工具。通过创建基于物理的详细真实空间模型，数字孪生使得在虚拟环境中训练机器学习算法成为可能，从而降低了物理测试的成本和限制。尽管具有这些优势，当前在数字孪生中训练的机器学习模型常常对特定虚拟环境过拟合，并且在应用于新场景时需要昂贵的重新训练。本文提出了一种物理信息强化学习方法，该方法利用数字孪生提供的物理洞察来塑造强化学习的奖励函数。通过将信号强度、到达角和路径反射等基于物理的指标整合到学习过程中，PIRL能够实现高效学习，并在无需重新训练的情况下提升对新环境的泛化能力。我们的实验表明，在数字孪生仿真的支持下，所提出的PIRL方法优于传统启发式算法和标准强化学习模型，在未见环境中实现了零样本泛化，为无线室内导航提供了一种经济高效、可扩展的解决方案。