Motion segmentation from a single moving camera presents a significant challenge in the field of computer vision. This challenge is compounded by the unknown camera movements and the lack of depth information of the scene. While deep learning has shown impressive capabilities in addressing these issues, supervised models require extensive training on massive annotated datasets, and unsupervised models also require training on large volumes of unannotated data, presenting significant barriers for both. In contrast, traditional methods based on optical flow do not require training data, however, they often fail to capture object-level information, leading to over-segmentation or under-segmentation. In addition, they also struggle in complex scenes with substantial depth variations and non-rigid motion, due to the overreliance of optical flow. To overcome these challenges, we propose an innovative hybrid approach that leverages the advantages of both deep learning methods and traditional optical flow based methods to perform dense motion segmentation without requiring any training. Our method initiates by automatically generating object proposals for each frame using foundation models. These proposals are then clustered into distinct motion groups using both optical flow and relative depth maps as motion cues. The integration of depth maps derived from state-of-the-art monocular depth estimation models significantly enhances the motion cues provided by optical flow, particularly in handling motion parallax issues. Our method is evaluated on the DAVIS-Moving and YTVOS-Moving datasets, and the results demonstrate that our method outperforms the best unsupervised method and closely matches with the state-of-theart supervised methods.

翻译：单目运动相机下的运动分割是计算机视觉领域的一项重大挑战。该挑战因相机运动未知及场景深度信息缺失而进一步加剧。尽管深度学习在解决这些问题上展现出卓越能力，但监督模型需要在大规模标注数据集上进行广泛训练，而无监督模型同样需要大量未标注数据进行训练，这为两者都带来了显著障碍。相比之下，基于光流的传统方法无需训练数据，但往往难以捕捉物体级信息，导致过分割或欠分割。此外，由于对光流的过度依赖，这些方法在深度变化剧烈和非刚性运动的复杂场景中也表现不佳。为克服这些挑战，我们提出一种创新的混合方法，该方法融合了深度学习方法和传统光流方法的优势，无需任何训练即可实现密集运动分割。我们的方法首先利用基础模型为每帧图像自动生成物体候选区域，随后结合光流与相对深度图作为运动线索，将这些候选区域聚类为不同的运动组。通过集成从最先进的单目深度估计模型导出的深度图，显著增强了光流提供的运动线索，特别是在处理运动视差问题方面。我们在DAVIS-Moving和YTVOS-Moving数据集上评估了本方法，结果表明其性能优于最佳无监督方法，并与最先进的监督方法表现相当。