Existing recurrent optical flow estimation networks are computationally expensive since they use a fixed large number of iterations to update the flow field for each sample. An efficient network should skip iterations when the flow improvement is limited. In this paper, we develop a Context-Aware Iteration Policy Network for efficient optical flow estimation, which determines the optimal number of iterations per sample. The policy network achieves this by learning contextual information to realize whether flow improvement is bottlenecked or minimal. On the one hand, we use iteration embedding and historical hidden cell, which include previous iterations information, to convey how flow has changed from previous iterations. On the other hand, we use the incremental loss to make the policy network implicitly perceive the magnitude of optical flow improvement in the subsequent iteration. Furthermore, the computational complexity in our dynamic network is controllable, allowing us to satisfy various resource preferences with a single trained model. Our policy network can be easily integrated into state-of-the-art optical flow networks. Extensive experiments show that our method maintains performance while reducing FLOPs by about 40%/20% for the Sintel/KITTI datasets.

翻译：现有递归光流估计网络计算开销较大，原因在于其对每个样本采用固定的大迭代次数来更新光流场。高效网络应在光流改进有限时跳过迭代步骤。本文提出一种面向高效光流估计的上下文感知迭代策略网络，可为每个样本确定最优迭代次数。该策略网络通过学习上下文信息来判断光流改进是否达到瓶颈或趋于极小。一方面，我们利用包含先前迭代信息的迭代嵌入与历史隐藏状态单元，来反映光流相对于先前迭代的变化趋势；另一方面，通过引入增量损失函数，使策略网络隐式感知后续迭代中光流改进的幅度。此外，我们设计的动态网络计算复杂度可控，单一训练模型即可满足不同资源偏好。该策略网络可轻松集成至当前最优光流网络中。大量实验表明，我们的方法在保持性能的同时，在Sintel/KITTI数据集上分别降低约40%/20%的FLOPs。