We present a novel camera path optimization framework for the task of online video stabilization. Typically, a stabilization pipeline consists of three steps: motion estimating, path smoothing, and novel view rendering. Most previous methods concentrate on motion estimation, proposing various global or local motion models. In contrast, path optimization receives relatively less attention, especially in the important online setting, where no future frames are available. In this work, we adopt recent off-the-shelf high-quality deep motion models for motion estimation to recover the camera trajectory and focus on the latter two steps. Our network takes a short 2D camera path in a sliding window as input and outputs the stabilizing warp field of the last frame in the window, which warps the coming frame to its stabilized position. A hybrid loss is well-defined to constrain the spatial and temporal consistency. In addition, we build a motion dataset that contains stable and unstable motion pairs for the training. Extensive experiments demonstrate that our approach significantly outperforms state-of-the-art online methods both qualitatively and quantitatively and achieves comparable performance to offline methods. Our code and dataset are available at https://github.com/liuzhen03/NNDVS

翻译：我们提出了一种新颖的相机路径优化框架，用于在线视频防抖任务。通常，防抖流程包含三个步骤：运动估计、路径平滑和新视角渲染。以往大多数方法集中于运动估计，提出了各种全局或局部运动模型。相比之下，路径优化受到的关注相对较少，尤其是在重要的在线设置中（此时无未来帧可用）。本工作中，我们采用近期现成的高质量深度运动模型进行运动估计，以恢复相机轨迹，并聚焦于后两个步骤。我们的网络以滑动窗口中的短时二维相机路径为输入，输出窗口内最后一帧的稳定化变形场，该变形场将后续帧映射至其稳定位置。我们定义了一种混合损失函数以约束空间与时间一致性。此外，我们构建了一个包含稳定与不稳定运动对的运动数据集用于训练。大量实验表明，我们的方法在定性与定量上均显著优于当前最先进的在线方法，并达到与离线方法可比的性能。我们的代码和数据集已在 https://github.com/liuzhen03/NNDVS 公开。