The majority of human detection methods rely on the sensor using visible lights (e.g., RGB cameras) but such sensors are limited in scenarios with degraded vision conditions. In this paper, we present a multimodal human detection system that combines portable thermal cameras and single-chip mmWave radars. To mitigate the noisy detection features caused by the low contrast of thermal cameras and the multi-path noise of radar point clouds, we propose a Bayesian feature extractor and a novel uncertainty-guided fusion method that surpasses a variety of competing methods, either single-modal or multi-modal. We evaluate the proposed method on real-world data collection and demonstrate that our approach outperforms the state-of-the-art methods by a large margin.

翻译：现有大多数人体检测方法依赖可见光传感器（例如 RGB 摄像头），但此类传感器在视觉退化条件下性能受限。本文提出一种多模态人体检测系统，融合便携式热成像摄像头与单芯片毫米波雷达。为抑制由热成像低对比度及雷达点云多径噪声引起的检测特征噪声，我们提出贝叶斯特征提取器及一种新颖的不确定性引导融合方法，其在单模态与多模态检测方案中均优于多种竞争方法。基于真实数据采集的实验表明，本方法在性能上大幅超越现有最优技术。