LiDAR has become one of the primary sensors in robotics and autonomous system for high-accuracy situational awareness. In recent years, multi-modal LiDAR systems emerged, and among them, LiDAR-as-a-camera sensors provide not only 3D point clouds but also fixed-resolution 360{\deg}panoramic images by encoding either depth, reflectivity, or near-infrared light in the image pixels. This potentially brings computer vision capabilities on top of the potential of LiDAR itself. In this paper, we are specifically interested in utilizing LiDARs and LiDAR-generated images for tracking Unmanned Aerial Vehicles (UAVs) in real-time which can benefit applications including docking, remote identification, or counter-UAV systems, among others. This is, to the best of our knowledge, the first work that explores the possibility of fusing the images and point cloud generated by a single LiDAR sensor to track a UAV without a priori known initialized position. We trained a custom YOLOv5 model for detecting UAVs based on the panoramic images collected in an indoor experiment arena with a MOCAP system. By integrating with the point cloud, we are able to continuously provide the position of the UAV. Our experiment demonstrated the effectiveness of the proposed UAV tracking approach compared with methods based only on point clouds or images. Additionally, we evaluated the real-time performance of our approach on the Nvidia Jetson Nano, a popular mobile computing platform.

翻译：激光雷达已成为机器人与自主系统中实现高精度态势感知的主要传感器之一。近年来，多模态激光雷达系统逐渐兴起，其中作为相机传感器的激光雷达不仅能够提供三维点云，还可通过将深度、反射率或近红外光编码至图像像素，生成固定分辨率的360度全景图像。这有望在激光雷达自身潜能之上叠加计算机视觉能力。本文专注于利用激光雷达及其生成的图像实时追踪无人飞行器（UAV），该技术可应用于自动对接、远程识别或反无人机系统等场景。据我们所知，这是首次探索融合单颗激光雷达生成的图像与点云数据、无需先验初始位置即可追踪无人机的研究工作。我们基于室内实验场地（搭载动作捕捉系统）采集的全景图像训练了定制化YOLOv5模型用于无人机检测。通过与点云数据集成，我们能够持续提供无人机的位置信息。实验证明，相比仅基于点云或仅基于图像的方法，本文提出的无人机追踪方法具有更优效果。此外，我们还在主流移动计算平台Nvidia Jetson Nano上评估了该方法的实时性能。