We propose a robust and fast bundle adjustment solution that estimates the 6-DoF pose of the camera and the geometry of the environment based on measurements from a rolling shutter (RS) camera. This tackles the challenges in the existing works, namely relying on additional sensors, high frame rate video as input, restrictive assumptions on camera motion, readout direction, and poor efficiency. To this end, we first investigate the influence of normalization to the image point on RSBA performance and show its better approximation in modelling the real 6-DoF camera motion. Then we present a novel analytical model for the visual residual covariance, which can be used to standardize the reprojection error during the optimization, consequently improving the overall accuracy. More importantly, the combination of normalization and covariance standardization weighting in RSBA (NW-RSBA) can avoid common planar degeneracy without needing to constrain the filming manner. Besides, we propose an acceleration strategy for NW-RSBA based on the sparsity of its Jacobian matrix and Schur complement. The extensive synthetic and real data experiments verify the effectiveness and efficiency of the proposed solution over the state-of-the-art works. We also demonstrate the proposed method can be easily implemented and plug-in famous GSSfM and GSSLAM systems as completed RSSfM and RSSLAM solutions.

翻译：我们提出了一种鲁棒且快速的集束调整解决方案，该方案基于卷帘快门相机的测量值，估计相机的六自由度位姿及环境几何结构。这克服了现有工作中的挑战，即依赖额外传感器、以高帧率视频作为输入、对相机运动及读出方向的限制性假设、以及效率低下等问题。为此，我们首先研究了图像点归一化对卷帘快门集束调整性能的影响，并展示了其在建模真实六自由度相机运动时更优的近似效果。随后，我们提出了一种全新的视觉残差协方差解析模型，该模型可在优化过程中标准化重投影误差，从而提升整体精度。更重要的是，卷帘快门集束调整中归一化与协方差标准化加权的结合，可在无需约束拍摄方式的前提下，避免常见的平面退化问题。此外，我们基于雅可比矩阵和舒尔补的稀疏性，提出了一种针对归一化加权卷帘快门集束调整的加速策略。大量合成与真实数据实验验证了所提方案相对于现有最先进工作的有效性与高效性。我们还证明了该方法易于实现，可作为完整的卷帘快门式结构从运动及卷帘快门式同时定位与地图构建解决方案，无缝集成至著名的通用结构从运动与通用同时定位与地图构建系统中。