Applying single image Monocular Depth Estimation (MDE) models to video sequences introduces significant temporal instability and flickering artifacts. We propose a novel approach that adapts any state-of-the-art image-based (depth) estimation model for video processing by integrating a new temporal module - trainable on a single GPU in a few days. Our architecture StableDPT builds upon an off-the-shelf Vision Transformer (ViT) encoder and enhances the Dense Prediction Transformer (DPT) head. The core of our contribution lies in the temporal layers within the head, which use an efficient cross-attention mechanism to integrate information from keyframes sampled across the entire video sequence. This allows the model to capture global context and inter-frame relationships leading to more accurate and temporally stable depth predictions. Furthermore, we propose a novel inference strategy for processing videos of arbitrary length avoiding the scale misalignment and redundant computations associated with overlapping windows used in other methods. Evaluations on multiple benchmark datasets demonstrate improved temporal consistency, competitive state-of-the-art performance and on top 2x faster processing in real-world scenarios.

翻译：将单幅图像的单目深度估计（MDE）模型应用于视频序列会引入显著的时序不稳定性和闪烁伪影。我们提出了一种新颖的方法，通过集成一个新的时序模块——可在单个GPU上数天内完成训练——来适配任何先进的基于图像的（深度）估计模型以进行视频处理。我们的架构StableDPT基于现成的Vision Transformer（ViT）编码器，并增强了Dense Prediction Transformer（DPT）头部。我们贡献的核心在于头部内的时序层，该层使用高效的交叉注意力机制来整合从整个视频序列中采样的关键帧信息。这使得模型能够捕获全局上下文和帧间关系，从而实现更准确且时序稳定的深度预测。此外，我们提出了一种新颖的推理策略，用于处理任意长度的视频，避免了其他方法中使用的重叠窗口所带来的尺度失准和冗余计算。在多个基准数据集上的评估表明，该方法在时序一致性方面有所提升，具有竞争力的先进性能，并且在真实场景中处理速度提高了至少2倍。