Aerial robots have the potential to play a crucial role in assisting humans with complex and dangerous tasks. Nevertheless, the future industry demands innovative solutions to streamline the interaction process between humans and drones to enable seamless collaboration and efficient co-working. In this paper, we present a novel tele-immersive framework that promotes cognitive and physical collaboration between humans and robots through Mixed Reality (MR). This framework incorporates a novel bi-directional spatial awareness and a multi-modal virtual-physical interaction approaches. The former seamlessly integrates the physical and virtual worlds, offering bidirectional egocentric and exocentric environmental representations. The latter, leveraging the proposed spatial representation, further enhances the collaboration combining a robot planning algorithm for obstacle avoidance with a variable admittance control. This allows users to issue commands based on virtual forces while maintaining compatibility with the environment map. We validate the proposed approach by performing several collaborative planning and exploration tasks involving a drone and an user equipped with a MR headset.

翻译：空中机器人在协助人类完成复杂且危险的任务方面具有巨大潜力。然而，未来工业需要创新解决方案来简化人与无人机之间的交互过程，以实现无缝协作和高效协同工作。本文提出了一种新型远程沉浸式框架，通过混合现实技术促进人与机器人之间的认知与物理协作。该框架融合了双向空间感知机制和多模态虚拟-物理交互方法：前者无缝整合物理世界与虚拟世界，提供双向的自我中心与外部环境表征；后者基于所提出的空间表征，结合用于避障的机器人规划算法与可变导纳控制，进一步增强了协作能力，使用户能够基于虚拟力发出指令，同时保持与环境地图的兼容性。我们通过执行多项涉及无人机与佩戴混合现实头显用户的协作规划及探索任务，验证了所提方法的有效性。