This paper presents a mixed-reality human-robot teaming system. It allows human operators to see in real-time where robots are located, even if they are not in line of sight. The operator can also visualize the map that the robots create of their environment and can easily send robots to new goal positions. The system mainly consists of a mapping and a control module. The mapping module is a real-time multi-agent visual SLAM system that co-localizes all robots and mixed-reality devices to a common reference frame. Visualizations in the mixed-reality device then allow operators to see a virtual life-sized representation of the cumulative 3D map overlaid onto the real environment. As such, the operator can effectively "see through" walls into other rooms. To control robots and send them to new locations, we propose a drag-and-drop interface. An operator can grab any robot hologram in a 3D mini map and drag it to a new desired goal pose. We validate the proposed system through a user study and real-world deployments. We make the mixed-reality application publicly available at https://github.com/cvg/HoloLens_ros.

翻译：本文提出了一种混合现实人机协同系统。该系统可使操作员实时掌握机器人位置信息，即使机器人处于视线盲区。操作员不仅能直观查看机器人构建的环境地图，还可便捷地向机器人发送新的目标位置指令。系统主要由建图模块与控制模块构成：建图模块采用实时多智能体视觉SLAM系统，将所有机器人及混合现实设备协同定位至统一参考坐标系；通过混合现实设备的可视化功能，操作员可观察到叠加于真实环境上的虚拟等比例三维累积地图，从而实现“透视墙壁”查看其他房间的效果。在机器人控制与路径规划方面，我们提出拖放式交互界面：操作员可在三维缩略地图中抓取任意机器人全息投影，将其拖拽至期望的目标位姿。通过用户实验与真实场景部署验证了该系统有效性，并将混合现实应用程序开源至https://github.com/cvg/HoloLens_ros。