Existing recurrent optical flow estimation networks are computationally expensive since they use a fixed large number of iterations to update the flow field for each sample. An efficient network should skip iterations when the flow improvement is limited. In this paper, we develop a Context-Aware Iteration Policy Network for efficient optical flow estimation, which determines the optimal number of iterations per sample. The policy network achieves this by learning contextual information to realize whether flow improvement is bottlenecked or minimal. On the one hand, we use iteration embedding and historical hidden cell, which include previous iterations information, to convey how flow has changed from previous iterations. On the other hand, we use the incremental loss to make the policy network implicitly perceive the magnitude of optical flow improvement in the subsequent iteration. Furthermore, the computational complexity in our dynamic network is controllable, allowing us to satisfy various resource preferences with a single trained model. Our policy network can be easily integrated into state-of-the-art optical flow networks. Extensive experiments show that our method maintains performance while reducing FLOPs by about 40%/20% for the Sintel/KITTI datasets.

翻译：现有循环光流估计网络因对每个样本采用固定的多次迭代更新光流场，导致计算代价高昂。高效网络应在流改进有限时跳过迭代。本文提出一种用于高效光流估计的上下文感知迭代策略网络，该网络可为每个样本确定最优迭代次数。策略网络通过学习上下文信息实现这一目标，以判断流改进是否遇瓶颈或趋于最小。一方面，我们利用包含先前迭代信息的迭代嵌入和历史隐藏单元来传递流如何自先前迭代发生变化。另一方面，我们采用增量损失函数使策略网络隐式感知后续迭代中光流改进的幅度。此外，我们动态网络的计算复杂度可控，使得单一训练模型可适配多种资源偏好。该策略网络可轻松集成到最先进的光流网络中。大量实验表明，我们的方法在保持性能的同时，对Sintel/KITTI数据集的FLOPs分别减少约40%/20%。