This paper introduces FlowMap, an end-to-end differentiable method that solves for precise camera poses, camera intrinsics, and per-frame dense depth of a video sequence. Our method performs per-video gradient-descent minimization of a simple least-squares objective that compares the optical flow induced by depth, intrinsics, and poses against correspondences obtained via off-the-shelf optical flow and point tracking. Alongside the use of point tracks to encourage long-term geometric consistency, we introduce differentiable re-parameterizations of depth, intrinsics, and pose that are amenable to first-order optimization. We empirically show that camera parameters and dense depth recovered by our method enable photo-realistic novel view synthesis on 360-degree trajectories using Gaussian Splatting. Our method not only far outperforms prior gradient-descent based bundle adjustment methods, but surprisingly performs on par with COLMAP, the state-of-the-art SfM method, on the downstream task of 360-degree novel view synthesis (even though our method is purely gradient-descent based, fully differentiable, and presents a complete departure from conventional SfM).

翻译：本文提出FlowMap，一种端到端可微方法，用于求解视频序列的精确相机位姿、内参及每帧密集深度。该方法通过逐视频梯度下降优化一个简单的最小二乘目标函数，该函数将深度、内参和位姿诱导的光流与现成光流和点跟踪获得的对应关系进行匹配。结合使用点跟踪以增强长期几何一致性，我们引入了深度、内参和位姿的可微重参数化方法，使其适用于一阶优化。实验表明，我们的方法恢复的相机参数和密集深度能够通过高斯泼溅实现360度轨迹上的照片级新视角合成。我们的方法不仅显著优于基于梯度下降的束调整方法，而且在360度新视角合成下游任务中与最先进的SfM方法COLMAP性能持平（尽管我们的方法完全基于梯度下降、完全可微，且与传统SfM方法存在根本性差异）。