LiDAR has become one of the primary sensors in robotics and autonomous system for high-accuracy situational awareness. In recent years, multi-modal LiDAR systems emerged, and among them, LiDAR-as-a-camera sensors provide not only 3D point clouds but also fixed-resolution 360{\deg}panoramic images by encoding either depth, reflectivity, or near-infrared light in the image pixels. This potentially brings computer vision capabilities on top of the potential of LiDAR itself. In this paper, we are specifically interested in utilizing LiDARs and LiDAR-generated images for tracking Unmanned Aerial Vehicles (UAVs) in real-time which can benefit applications including docking, remote identification, or counter-UAV systems, among others. This is, to the best of our knowledge, the first work that explores the possibility of fusing the images and point cloud generated by a single LiDAR sensor to track a UAV without a priori known initialized position. We trained a custom YOLOv5 model for detecting UAVs based on the panoramic images collected in an indoor experiment arena with a MOCAP system. By integrating with the point cloud, we are able to continuously provide the position of the UAV. Our experiment demonstrated the effectiveness of the proposed UAV tracking approach compared with methods based only on point clouds or images. Additionally, we evaluated the real-time performance of our approach on the Nvidia Jetson Nano, a popular mobile computing platform.

翻译：激光雷达已成为机器人和自主系统中用于高精度态势感知的主要传感器之一。近年来，多模态激光雷达系统涌现，其中，作为相机传感器的激光雷达不仅提供三维点云，还通过将深度、反射率或近红外光编码到图像像素中，生成固定分辨率的360°全景图像。这有望在激光雷达自身潜力的基础上赋予计算机视觉能力。本文特别关注利用激光雷达及其生成的图像实时跟踪无人飞行器，这可用于无人机对接、远程识别或反无人机系统等应用。据我们所知，这是首次探索融合单一激光雷达传感器生成的图像与点云，在无需预先已知初始化位置的情况下跟踪无人机。我们基于室内实验场地中由运动捕捉系统采集的全景图像，训练了定制的YOLOv5模型用于检测无人机。通过与点云集成，我们能够连续提供无人机的位置。实验表明，与仅基于点云或图像的方法相比，所提出的无人机跟踪方法具有有效性。此外，我们在流行的移动计算平台Nvidia Jetson Nano上评估了该方法的实时性能。