As mobile robots increasingly operate alongside humans in shared workspaces, ensuring safe, efficient, and interpretable Human-Robot Interaction (HRI) has become a pressing challenge. While substantial progress has been devoted to human behavior prediction, limited attention has been paid to how humans perceive, interpret, and trust robots' inferences, impeding deployment in safety-critical and socially embedded environments. This paper presents XR-DT, an eXtended Reality-enhanced Digital Twin framework for agentic mobile robots, that bridges physical and virtual spaces to enable bi-directional understanding between humans and robots. Our hierarchical XR-DT architecture integrates virtual-, augmented-, and mixed-reality layers, fusing real-time sensor data, simulated environments in the Unity game engine, and human feedback captured through wearable AR devices. Within this framework, we design an agentic mobile robot system with a unified diffusion policy for context-aware task adaptation. We further propose a chain-of-thought prompting mechanism that allows multimodal large language models to reason over human instructions and environmental context, while leveraging an AutoGen-based multi-agent coordination layer to enhance robustness and collaboration in dynamic tasks. Initial experimental results demonstrate accurate human and robot trajectory prediction, validating the XR-DT framework's effectiveness in HRI tasks. By embedding human intention, environmental dynamics, and robot cognition into the XR-DT framework, our system enables interpretable, trustworthy, and adaptive HRI.

翻译：随着移动机器人在与人类共享的工作空间中日益广泛地运行，确保安全、高效且可解释的人机交互已成为一项紧迫挑战。尽管在人类行为预测方面已取得显著进展，但人类如何感知、解释并信任机器人的推断却鲜有研究，这阻碍了机器人在安全关键和社会化嵌入环境中的部署。本文提出XR-DT，一种面向智能体移动机器人的扩展现实增强数字孪生框架，该框架通过连接物理与虚拟空间，实现人机双向理解。我们的分层XR-DT架构整合了虚拟现实、增强现实与混合现实层，融合了实时传感器数据、Unity游戏引擎中的仿真环境以及通过可穿戴AR设备捕获的人类反馈。在此框架内，我们设计了一个采用统一扩散策略以实现情境感知任务自适应的智能体移动机器人系统。进一步提出一种思维链提示机制，使多模态大语言模型能够对人类指令与环境情境进行推理，同时利用基于AutoGen的多智能体协调层增强动态任务中的鲁棒性与协作能力。初步实验结果表明，该系统能准确预测人类与机器人轨迹，验证了XR-DT框架在人机交互任务中的有效性。通过将人类意图、环境动态与机器人认知嵌入XR-DT框架，我们的系统实现了可解释、可信赖且自适应的人机交互。