Optical Flow Estimation aims to find the 2D dense motion field between two frames. Due to the limitation of model structures and training datasets, existing methods often rely too much on local clues and ignore the integrity of objects, resulting in fragmented motion estimation. Through theoretical analysis, we find the pre-trained large vision models are helpful in optical flow estimation, and we notice that the recently famous Segment Anything Model (SAM) demonstrates a strong ability to segment complete objects, which is suitable for solving the fragmentation problem. We thus propose a solution to embed the frozen SAM image encoder into FlowFormer to enhance object perception. To address the challenge of in-depth utilizing SAM in non-segmentation tasks like optical flow estimation, we propose an Optical Flow Task-Specific Adaption scheme, including a Context Fusion Module to fuse the SAM encoder with the optical flow context encoder, and a Context Adaption Module to adapt the SAM features for optical flow task with Learned Task-Specific Embedding. Our proposed SAMFlow model reaches 0.86/2.10 clean/final EPE and 3.55/12.32 EPE/F1-all on Sintel and KITTI-15 training set, surpassing Flowformer by 8.5%/9.9% and 13.2%/16.3%. Furthermore, our model achieves state-of-the-art performance on the Sintel and KITTI-15 benchmarks, ranking #1 among all two-frame methods on Sintel clean pass.

翻译：光流估计旨在寻找两帧图像之间的二维稠密运动场。受限于模型结构和训练数据集，现有方法往往过度依赖局部线索而忽略物体完整性，导致运动估计出现碎片化。通过理论分析，我们发现预训练大型视觉模型有助于光流估计，并注意到近期著名的分割一切模型（SAM）展现出强大的完整物体分割能力，适用于解决碎片化问题。因此，我们提出将冻结的SAM图像编码器嵌入FlowFormer以增强物体感知的方案。为解决在光流估计等非分割任务中深度利用SAM的挑战，我们提出了一种光流任务特定适配方案，包括上下文融合模块（将SAM编码器与光流上下文编码器融合）和上下文适配模块（通过学习的任务特定嵌入将SAM特征适配到光流任务）。我们提出的SAMFlow模型在Sintel和KITTI-15训练集上达到0.86/2.10的clean/final EPE和3.55/12.32的EPE/F1-all，相比Flowformer分别提升8.5%/9.9%和13.2%/16.3%。此外，我们的模型在Sintel和KITTI-15基准测试中达到最先进性能，在Sintel clean通道中位列所有双帧方法第一。