In this work, we propose a novel Mamba block DenVisCoM, as well as a novel hybrid architecture specifically tailored for accurate and real-time estimation of optical flow and disparity estimation. Given that such multi-view geometry and motion tasks are fundamentally related, we propose a unified architecture to tackle them jointly. Specifically, the proposed hybrid architecture is based on DenVisCoM and a Transformer-based attention block that efficiently addresses real-time inference, memory footprint, and accuracy at the same time for joint estimation of motion and 3D dense perception tasks. We extensively analyze the benchmark trade-off of accuracy and real-time processing on a large number of datasets. Our experimental results and related analysis suggest that our proposed model can accurately estimate optical flow and disparity estimation in real time. All models and associated code are available at https://github.com/vimstereo/DenVisCoM.

翻译：本文提出了一种新颖的Mamba模块DenVisCoM，以及一种专为精确、实时估计光流与视差而设计的新型混合架构。鉴于此类多视图几何与运动任务本质上是相互关联的，我们提出了一种统一的架构来联合处理它们。具体而言，所提出的混合架构基于DenVisCoM和一个基于Transformer的注意力模块，该模块能够同时高效地解决实时推理、内存占用和精度问题，以完成运动与三维稠密感知任务的联合估计。我们在大量数据集上深入分析了精度与实时处理的基准权衡。我们的实验结果及相关分析表明，所提出的模型能够实时准确地估计光流与视差。所有模型及相关代码均可在 https://github.com/vimstereo/DenVisCoM 获取。