Key-point-based scene understanding is fundamental for autonomous driving applications. At the same time, optical flow plays an important role in many vision tasks. However, due to the implicit bias of equal attention on all points, classic data-driven optical flow estimation methods yield less satisfactory performance on key points, limiting their implementations in key-point-critical safety-relevant scenarios. To address these issues, we introduce a points-based modeling method that requires the model to learn key-point-related priors explicitly. Based on the modeling method, we present FocusFlow, a framework consisting of 1) a mix loss function combined with a classic photometric loss function and our proposed Conditional Point Control Loss (CPCL) function for diverse point-wise supervision; 2) a conditioned controlling model which substitutes the conventional feature encoder by our proposed Condition Control Encoder (CCE). CCE incorporates a Frame Feature Encoder (FFE) that extracts features from frames, a Condition Feature Encoder (CFE) that learns to control the feature extraction behavior of FFE from input masks containing information of key points, and fusion modules that transfer the controlling information between FFE and CFE. Our FocusFlow framework shows outstanding performance with up to +44.5% precision improvement on various key points such as ORB, SIFT, and even learning-based SiLK, along with exceptional scalability for most existing data-driven optical flow methods like PWC-Net, RAFT, and FlowFormer. Notably, FocusFlow yields competitive or superior performances rivaling the original models on the whole frame. The source code will be available at https://github.com/ZhonghuaYi/FocusFlow_official.

翻译：基于关键点的场景理解是自动驾驶应用的基础。同时，光流在众多视觉任务中发挥着重要作用。然而，由于对所有点等量关注的隐式偏差，经典数据驱动光流估计方法在关键点上表现欠佳，限制了其在关键点至关重要的安全相关场景中的应用。为解决这些问题，我们提出了一种基于点的建模方法，要求模型显式学习关键点相关先验信息。基于该建模方法，我们提出了FocusFlow框架，该框架包含：1）一种混合损失函数，结合了经典光度损失函数与本文提出的条件点控制损失（CPCL）函数，用于多样化的逐点监督；2）一个条件控制模型，用本文提出的条件控制编码器（CCE）替代传统特征编码器。CCE包含用于提取帧特征的帧特征编码器（FFE）、通过包含关键点信息的输入掩码学习控制FFE特征提取行为的条件特征编码器（CFE），以及用于在FFE和CFE之间传递控制信息的融合模块。FocusFlow框架在ORB、SIFT乃至基于学习的SiLK等多种关键点上展现出卓越性能，精度提升高达44.5%，并且对现有大多数数据驱动光流方法（如PWC-Net、RAFT和FlowFormer）具有出色的可扩展性。值得注意的是，FocusFlow在全帧上的性能与原始模型相比具有竞争力或更优。源代码将发布在https://github.com/ZhonghuaYi/FocusFlow_official。