In this paper, we study state estimation of multi-visual-inertial systems (MVIS) and develop sensor fusion algorithms to optimally fuse an arbitrary number of asynchronous inertial measurement units (IMUs) or gyroscopes and global and(or) rolling shutter cameras. We are especially interested in the full calibration of the associated visual-inertial sensors, including the IMU or camera intrinsics and the IMU-IMU(or camera) spatiotemporal extrinsics as well as the image readout time of rolling-shutter cameras (if used). To this end, we develop a new analytic combined IMU integration with intrinsics-termed ACI3-to preintegrate IMU measurements, which is leveraged to fuse auxiliary IMUs and(or) gyroscopes alongside a base IMU. We model the multi-inertial measurements to include all the necessary inertial intrinsic and IMU-IMU spatiotemporal extrinsic parameters, while leveraging IMU-IMU rigid-body constraints to eliminate the necessity of auxiliary inertial poses and thus reducing computational complexity. By performing observability analysis of MVIS, we prove that the standard four unobservable directions remain - no matter how many inertial sensors are used, and also identify, for the first time, degenerate motions for IMU-IMU spatiotemporal extrinsics and auxiliary inertial intrinsics. In addition to the extensive simulations that validate our analysis and algorithms, we have built our own MVIS sensor rig and collected over 25 real-world datasets to experimentally verify the proposed calibration against the state-of-the-art calibration method such as Kalibr. We show that the proposed MVIS calibration is able to achieve competing accuracy with improved convergence and repeatability, which is open sourced to better benefit the community.

翻译：本文研究了多视觉-惯性系统（MVIS）的状态估计问题，提出了传感器融合算法，能够最优融合任意数量的异步惯性测量单元（IMU）或陀螺仪，以及全局快门和/或卷帘快门相机。我们特别关注相关视觉-惯性传感器的完整标定，包括IMU或相机的内参、IMU-IMU（或相机）的时空外参，以及卷帘快门相机的图像读出时间（如使用）。为此，我们提出了名为ACI3的新型解析组合IMU积分方法，用于预积分IMU测量值，从而在基IMU之外融合辅助IMU和/或陀螺仪。我们对多惯性测量值进行建模，包含了所有必要的惯性内参和IMU-IMU时空外参参数，同时利用IMU-IMU刚体约束消除了辅助惯性位姿的必要性，从而降低了计算复杂度。通过MVIS的可观测性分析，我们证明了无论使用多少个惯性传感器，标准的四个不可观测方向保持不变，并且首次识别了IMU-IMU时空外参和辅助惯性内参的退化运动。除了通过大量仿真验证我们的分析和算法外，我们还搭建了自研MVIS传感器平台，收集了超过25个真实世界数据集，与Kalibr等最先进的标定方法进行了对比实验验证。实验表明，所提出的MVIS标定方法能够在改进的收敛性和重复性条件下达到相当的精度，并已开源以更好地惠及学术界。