Tracking subjects in videos is one of the most widely used functions in camera-based IoT applications such as security surveillance, smart city traffic safety enhancement, vehicle to pedestrian communication and so on. In the computer vision domain, tracking is usually achieved by first detecting subjects with bounding boxes, then associating detected bounding boxes across video frames. For many IoT systems, images captured by cameras are usually sent over the network to be processed at a different site that has more powerful computing resources than edge devices. However, sending entire frames through the network causes significant bandwidth consumption that may exceed the system bandwidth constraints. To tackle this problem, we propose ViFiT, a transformer-based model that reconstructs vision bounding box trajectories from phone data (IMU and Fine Time Measurements). It leverages a transformer ability of better modeling long-term time series data. ViFiT is evaluated on Vi-Fi Dataset, a large-scale multimodal dataset in 5 diverse real world scenes, including indoor and outdoor environments. To fill the gap of proper metrics of jointly capturing the system characteristics of both tracking quality and video bandwidth reduction, we propose a novel evaluation framework dubbed Minimum Required Frames (MRF) and Minimum Required Frames Ratio (MRFR). ViFiT achieves an MRFR of 0.65 that outperforms the state-of-the-art approach for cross-modal reconstruction in LSTM Encoder-Decoder architecture X-Translator of 0.98, resulting in a high frame reduction rate as 97.76%.

翻译：视频中目标跟踪是摄像头物联网系统（如安全监控、智慧城市交通安全增强、车与行人通信等）中最广泛使用的功能之一。在计算机视觉领域，跟踪通常通过首先用边界框检测目标，然后在视频帧间关联检测到的边界框来实现。对于许多物联网系统，摄像头采集的图像通常通过网络发送到边缘设备以外具有更强计算资源的站点进行处理。然而，通过网络传输完整帧会导致显著的带宽消耗，这可能超出系统带宽限制。为解决此问题，我们提出ViFiT——一种基于Transformer的模型，能够从手机数据（IMU和细粒度时间测量）重建视觉边界框轨迹。它利用Transformer更好地建模长期时序数据的能力。ViFiT在Vi-Fi数据集上进行评估，该数据集是一个包含室内和室外等5种不同真实场景的大规模多模态数据集。为填补联合捕捉跟踪质量和视频带宽缩减系统特性的适当度量空白，我们提出了一种新的评估框架，称为最小所需帧数（MRF）和最小所需帧数比（MRFR）。ViFiT实现的MRFR为0.65，优于基于LSTM编解码器架构的最新跨模态重建方法X-Translator的0.98，实现了高达97.76%的帧缩减率。