In this work, we seek to predict camera poses across scenes with a multi-task learning manner, where we view the localization of each scene as a new task. We propose OFVL-MS, a unified framework that dispenses with the traditional practice of training a model for each individual scene and relieves gradient conflict induced by optimizing multiple scenes collectively, enabling efficient storage yet precise visual localization for all scenes. Technically, in the forward pass of OFVL-MS, we design a layer-adaptive sharing policy with a learnable score for each layer to automatically determine whether the layer is shared or not. Such sharing policy empowers us to acquire task-shared parameters for a reduction of storage cost and task-specific parameters for learning scene-related features to alleviate gradient conflict. In the backward pass of OFVL-MS, we introduce a gradient normalization algorithm that homogenizes the gradient magnitude of the task-shared parameters so that all tasks converge at the same pace. Furthermore, a sparse penalty loss is applied on the learnable scores to facilitate parameter sharing for all tasks without performance degradation. We conduct comprehensive experiments on multiple benchmarks and our new released indoor dataset LIVL, showing that OFVL-MS families significantly outperform the state-of-the-arts with fewer parameters. We also verify that OFVL-MS can generalize to a new scene with much few parameters while gaining superior localization performance.

翻译：本文以多任务学习方式预测跨场景的相机位姿，将每个场景的定位视为一个新任务。我们提出统一框架OFVL-MS，摒弃了为每个场景单独训练模型的传统做法，并通过缓解多场景联合优化引发的梯度冲突，实现了对所有场景的高效存储与精确视觉定位。在OFVL-MS的前向传播中，我们设计了一种层自适应共享策略，为每层分配可学习分数以自动决定该层是否共享。该策略使得任务共享参数可降低存储成本，而任务特定参数则用于学习场景相关特征以缓解梯度冲突。在反向传播中，我们引入梯度归一化算法，使任务共享参数的梯度量级均匀化，从而确保所有任务以相同速度收敛。此外，我们对可学习分数施加稀疏惩罚损失，以促进所有任务的参数共享且不损失性能。我们在多个基准数据集及新发布的室内数据集LIVL上进行了全面实验，结果表明OFVL-MS系列模型以更少参数显著超越现有最优方法。我们还验证了OFVL-MS能以极少的参数泛化至新场景，同时获得优越的定位性能。