High-definition (HD) cameras for surveillance and road traffic have experienced tremendous growth, demanding intensive computation resources for real-time analytics. Recently, offloading frames from the front-end device to the back-end edge server has shown great promise. In multi-stream competitive environments, efficient bandwidth management and proper scheduling are crucial to ensure both high inference accuracy and high throughput. To achieve this goal, we propose BiSwift, a bi-level framework that scales the concurrent real-time video analytics by a novel adaptive hybrid codec integrated with multi-level pipelines, and a global bandwidth controller for multiple video streams. The lower-level front-back-end collaborative mechanism (called adaptive hybrid codec) locally optimizes the accuracy and accelerates end-to-end video analytics for a single stream. The upper-level scheduler aims to accuracy fairness among multiple streams via the global bandwidth controller. The evaluation of BiSwift shows that BiSwift is able to real-time object detection on 9 streams with an edge device only equipped with an NVIDIA RTX3070 (8G) GPU. BiSwift improves 10%$\sim$21% accuracy and presents 1.2$\sim$9$\times$ throughput compared with the state-of-the-art video analytics pipelines.

翻译：高清（HD）监控与交通摄像头数量急剧增长，对实时分析所需的计算资源提出了大量需求。近年来，将视频帧从前端设备卸载至后端边缘服务器的方案展现出巨大潜力。在多流竞争环境中，高效的带宽管理与合理的调度策略对于同时保障高推理精度和高吞吐量至关重要。为此，我们提出BiSwift——一种双层框架：通过集成多级流水线的自适应混合编解码器实现并发实时视频分析的扩展，并采用全局带宽控制器管理多视频流。底层采用前端-后端协同机制（称为自适应混合编解码器），针对单流本地优化精度并加速端到端视频分析；上层调度器通过全局带宽控制器实现多流间的精度公平性。实验评估表明，BiSwift可在仅配备NVIDIA RTX3070（8G）GPU的边缘设备上对9路视频流进行实时目标检测。与当前最先进的视频分析流水线相比，BiSwift将精度提升10%∼21%，吞吐量提升1.2∼9倍。