Non-line-of-sight (NLOS) tracking has drawn increasing attention in recent years, due to its ability to detect object motion out of sight. Most previous works on NLOS tracking rely on active illumination, e.g., laser, and suffer from high cost and elaborate experimental conditions. Besides, these techniques are still far from practical application due to oversimplified settings. In contrast, we propose a purely passive method to track a person walking in an invisible room by only observing a relay wall, which is more in line with real application scenarios, e.g., security. To excavate imperceptible changes in videos of the relay wall, we introduce difference frames as an essential carrier of temporal-local motion messages. In addition, we propose PAC-Net, which consists of alternating propagation and calibration, making it capable of leveraging both dynamic and static messages on a frame-level granularity. To evaluate the proposed method, we build and publish the first dynamic passive NLOS tracking dataset, NLOS-Track, which fills the vacuum of realistic NLOS datasets. NLOS-Track contains thousands of NLOS video clips and corresponding trajectories. Both real-shot and synthetic data are included.
翻译:不可见区域(NLOS)跟踪因能检测视线外的物体运动,近年来受到广泛关注。以往大多数NLOS跟踪工作依赖主动照明(如激光),存在成本高、实验条件苛刻等局限。此外,由于设置过于简化,这些技术仍远未达到实际应用水平。与此相反,我们提出了一种纯被动方法,仅通过观察反射墙即可跟踪隐身房间内行走的人,这更符合实际应用场景(例如安防)。为了挖掘反射墙视频中难以察觉的变化,我们引入差分帧作为时空局部运动信息的关键载体。此外,我们提出PAC-Net,该网络由交替的传播与校准模块构成,能够在帧级粒度上同时利用动态与静态信息。为评估所提方法,我们构建并发布了首个动态被动NLOS跟踪数据集NLOS-Track,填补了真实NLOS数据集的空白。NLOS-Track包含数千个NLOS视频片段及对应轨迹,涵盖实拍与合成数据。