Human-robot collaboration (HRC) has become increasingly relevant in industrial, household, and commercial settings. However, the effectiveness of such collaborations is highly dependent on the human and robots' situational awareness of the environment. Improving this awareness includes not only aligning perceptions in a shared workspace, but also bidirectionally communicating intent and visualizing different states of the environment to enhance scene understanding. In this paper, we propose ARDIE (Augmented Reality with Dialogue and Eye Gaze), a novel intelligent agent that leverages multi-modal feedback cues to enhance HRC. Our system utilizes a decision theoretic framework to formulate a joint policy that incorporates interactive augmented reality (AR), natural language, and eye gaze to portray current and future states of the environment. Through object-specific AR renders, the human can visualize future object interactions to make adjustments as needed, ultimately providing an interactive and efficient collaboration between humans and robots.

翻译：人机协作在工业、家庭及商业场景中日益重要。然而，此类协作的有效性高度依赖于人与机器人对环境的态势感知能力。提升这种感知不仅需要共享工作空间中的感知对齐，还需双向沟通意图并可视化环境的不同状态以增强场景理解。本文提出ARDIE（增强现实结合对话与视线注视），一种利用多模态反馈线索增强人机协作的新型智能体。本系统采用决策理论框架构建联合策略，融合交互式增强现实、自然语言及视线注视，以呈现环境当前与未来状态。通过对象级AR渲染，人类可可视化预期物体交互并做出相应调整，最终实现人与机器人之间高效且具交互性的协作。