Aerial robots have the potential to play a crucial role in assisting humans with complex and dangerous tasks. Nevertheless, the future industry demands innovative solutions to streamline the interaction process between humans and drones to enable seamless collaboration and efficient co-working. In this paper, we present a novel tele-immersive framework that promotes cognitive and physical collaboration between humans and robots through Mixed Reality (MR). This framework incorporates a novel bi-directional spatial awareness and a multi-modal virtual-physical interaction approaches. The former seamlessly integrates the physical and virtual worlds, offering bidirectional egocentric and exocentric environmental representations. The latter, leveraging the proposed spatial representation, further enhances the collaboration combining a robot planning algorithm for obstacle avoidance with a variable admittance control. This allows users to issue commands based on virtual forces while maintaining compatibility with the environment map. We validate the proposed approach by performing several collaborative planning and exploration tasks involving a drone and an user equipped with a MR headset.

翻译：空中机器人在协助人类完成复杂危险任务方面具有重要作用。然而，未来工业需要创新解决方案来简化人与无人机之间的交互过程，以实现无缝协作和高效共事。本文提出一种新颖的远程沉浸式框架，通过混合现实（MR）促进人与机器人的认知与物理协作。该框架融合了新型双向空间感知和多模态虚实交互方法：前者无缝整合物理与虚拟世界，提供双向的自我中心与环境中心环境表征；后者则利用所提出的空间表征，结合避障的机器人规划算法与可变导纳控制进一步强化协作，允许用户基于虚拟力发布指令，同时保持与环境地图的兼容性。我们通过无人机与佩戴MR头显的用户执行多项协作规划与探索任务，验证了所提方法的有效性。