Existing recurrent optical flow estimation networks are computationally expensive since they use a fixed large number of iterations to update the flow field for each sample. An efficient network should skip iterations when the flow improvement is limited. In this paper, we develop a Context-Aware Iteration Policy Network for efficient optical flow estimation, which determines the optimal number of iterations per sample. The policy network achieves this by learning contextual information to realize whether flow improvement is bottlenecked or minimal. On the one hand, we use iteration embedding and historical hidden cell, which include previous iterations information, to convey how flow has changed from previous iterations. On the other hand, we use the incremental loss to make the policy network implicitly perceive the magnitude of optical flow improvement in the subsequent iteration. Furthermore, the computational complexity in our dynamic network is controllable, allowing us to satisfy various resource preferences with a single trained model. Our policy network can be easily integrated into state-of-the-art optical flow networks. Extensive experiments show that our method maintains performance while reducing FLOPs by about 40%/20% for the Sintel/KITTI datasets.

翻译：现有递归光流估计网络计算成本高昂，因为它们对每个样本使用固定的多次迭代来更新光流场。高效的网络应在流改进有限时跳过迭代。本文提出了一种面向高效光流估计的上下文感知迭代策略网络，该网络能为每个样本确定最优迭代次数。策略网络通过学习上下文信息来实现这一目标，从而判断流改进是否遇到瓶颈或已降至最低。一方面，我们利用包含先前迭代信息的迭代嵌入和历史隐藏单元来传达流如何从先前迭代中变化。另一方面，我们使用增量损失使策略网络隐式感知后续迭代中光流改进的幅度。此外，我们动态网络的计算复杂度是可控的，允许我们使用单个训练模型满足各种资源偏好。我们的策略网络可以轻松集成到最先进的光流网络中。大量实验表明，我们的方法在保持性能的同时，将Sintel/KITTI数据集的FLOPs减少了约40%/20%。