Video Anomaly Detection (VAD) aims to locate unusual activities or behaviors within videos. Recently, offline VAD has garnered substantial research attention, which has been invigorated by the progress in large language models (LLMs) and vision-language models (VLMs), offering the potential for a more nuanced understanding of anomalies. However, online VAD has seldom received attention due to real-time constraints and computational intensity. In this paper, we introduce a novel Memory-based online scoring queue scheme for Training-free VAD (MoniTor), to address the inherent complexities in online VAD. Specifically, MoniTor applies a streaming input to VLMs, leveraging the capabilities of pre-trained large-scale models. To capture temporal dependencies more effectively, we incorporate a novel prediction mechanism inspired by Long Short-Term Memory (LSTM) networks. This ensures the model can effectively model past states and leverage previous predictions to identify anomalous behaviors. Thereby, it better understands the current frame. Moreover, we design a scoring queue and an anomaly prior to dynamically store recent scores and cover all anomalies in the monitoring scenario, providing guidance for LLMs to distinguish between normal and abnormal behaviors over time. We evaluate MoniTor on two large datasets (i.e., UCF-Crime and XD-Violence) containing various surveillance and real-world scenarios. The results demonstrate that MoniTor outperforms state-of-the-art methods and is competitive with weakly supervised methods without training. Code is available at https://github.com/YsTvT/MoniTor.
翻译:视频异常检测(VAD)旨在定位视频中的异常活动或行为。近年来,离线VAD获得了大量研究关注,这得益于大型语言模型(LLMs)和视觉语言模型(VLMs)的进展,为更细致地理解异常提供了可能。然而,由于实时性约束和计算密集性,在线VAD很少受到关注。本文提出了一种新颖的基于记忆的在线评分队列方案,用于实现免训练的VAD(MoniTor),以应对在线VAD固有的复杂性。具体而言,MoniTor将流式输入应用于VLMs,充分利用预训练大规模模型的能力。为了更有效地捕捉时间依赖性,我们引入了一种受长短期记忆(LSTM)网络启发的新型预测机制。这确保了模型能够有效建模过去状态,并利用先前的预测来识别异常行为,从而更好地理解当前帧。此外,我们设计了一个评分队列和一个异常先验,动态存储近期评分并覆盖监控场景中的所有异常,为LLMs随时间区分正常与异常行为提供指导。我们在两个包含各种监控和真实场景的大型数据集(即UCF-Crime和XD-Violence)上评估MoniTor。结果表明,MoniTor优于现有最先进方法,且与无需训练的弱监督方法相比具有竞争力。代码可在https://github.com/YsTvT/MoniTor获取。