Key-point-based scene understanding is fundamental for autonomous driving applications. At the same time, optical flow plays an important role in many vision tasks. However, due to the implicit bias of equal attention on all points, classic data-driven optical flow estimation methods yield less satisfactory performance on key points, limiting their implementations in key-point-critical safety-relevant scenarios. To address these issues, we introduce a points-based modeling method that requires the model to learn key-point-related priors explicitly. Based on the modeling method, we present FocusFlow, a framework consisting of 1) a mix loss function combined with a classic photometric loss function and our proposed Conditional Point Control Loss (CPCL) function for diverse point-wise supervision; 2) a conditioned controlling model which substitutes the conventional feature encoder by our proposed Condition Control Encoder (CCE). CCE incorporates a Frame Feature Encoder (FFE) that extracts features from frames, a Condition Feature Encoder (CFE) that learns to control the feature extraction behavior of FFE from input masks containing information of key points, and fusion modules that transfer the controlling information between FFE and CFE. Our FocusFlow framework shows outstanding performance with up to +44.5% precision improvement on various key points such as ORB, SIFT, and even learning-based SiLK, along with exceptional scalability for most existing data-driven optical flow methods like PWC-Net, RAFT, and FlowFormer. Notably, FocusFlow yields competitive or superior performances rivaling the original models on the whole frame. The source code will be available at https://github.com/ZhonghuaYi/FocusFlow_official.

翻译：基于关键点的场景理解是自动驾驶应用的基础，同时光流在诸多视觉任务中扮演着重要角色。然而，由于经典数据驱动光流估计方法对所有点赋予同等关注的隐性偏差，导致其在关键点上性能欠佳，从而限制了其在关键点敏感的安全相关场景中的部署。针对这些问题，我们提出一种基于点的建模方法，要求模型显式学习与关键点相关的先验知识。基于该建模方法，我们提出FocusFlow框架，该框架包含：1）混合损失函数，结合经典光度损失函数与本文提出的条件点控制损失（CPCL）函数，实现多样化的逐点监督；2）条件控制模型，以本文提出的条件控制编码器（CCE）替代传统特征编码器。CCE包含帧特征编码器（FFE）（用于从帧中提取特征）、条件特征编码器（CFE）（学习从包含关键点信息的输入掩码中控制FFE特征提取行为），以及融合模块（在FFE与CFE之间传递控制信息）。我们的FocusFlow框架在ORB、SIFT乃至基于学习的SiLK等多种关键点上展现出卓越性能，精度提升高达44.5%，并具有出色的可扩展性，可适配PWC-Net、RAFT、FlowFormer等现有主流数据驱动光流方法。值得注意的是，FocusFlow在全帧上的性能与原始模型相比具有竞争力甚至更优。源代码将发布于https://github.com/ZhonghuaYi/FocusFlow_official。