In this paper, we study state estimation of multi-visual-inertial systems (MVIS) and develop sensor fusion algorithms to optimally fuse an arbitrary number of asynchronous inertial measurement units (IMUs) or gyroscopes and global and(or) rolling shutter cameras. We are especially interested in the full calibration of the associated visual-inertial sensors, including the IMU or camera intrinsics and the IMU-IMU(or camera) spatiotemporal extrinsics as well as the image readout time of rolling-shutter cameras (if used). To this end, we develop a new analytic combined IMU integration with intrinsics-termed ACI3-to preintegrate IMU measurements, which is leveraged to fuse auxiliary IMUs and(or) gyroscopes alongside a base IMU. We model the multi-inertial measurements to include all the necessary inertial intrinsic and IMU-IMU spatiotemporal extrinsic parameters, while leveraging IMU-IMU rigid-body constraints to eliminate the necessity of auxiliary inertial poses and thus reducing computational complexity. By performing observability analysis of MVIS, we prove that the standard four unobservable directions remain - no matter how many inertial sensors are used, and also identify, for the first time, degenerate motions for IMU-IMU spatiotemporal extrinsics and auxiliary inertial intrinsics. In addition to the extensive simulations that validate our analysis and algorithms, we have built our own MVIS sensor rig and collected over 25 real-world datasets to experimentally verify the proposed calibration against the state-of-the-art calibration method such as Kalibr. We show that the proposed MVIS calibration is able to achieve competing accuracy with improved convergence and repeatability, which is open sourced to better benefit the community.

翻译：本文研究了多视觉惯性系统（MVIS）的状态估计问题，并开发了传感器融合算法，以最优方式融合任意数量的异步惯性测量单元（IMU）或陀螺仪，以及全局和/或卷帘快门相机。我们特别关注相关视觉惯性传感器的完整标定，包括IMU或相机内参、IMU-IMU（或相机）时空外参，以及卷帘快门相机的图像读出时间（若使用）。为此，我们提出了一种新的解析组合IMU积分方法（称为ACI³）来预积分IMU测量值，该方法被用于在主IMU的基础上融合辅助IMU和/或陀螺仪。我们对多惯性测量进行建模，包含所有必要的惯性内参和IMU-IMU时空外参参数，同时利用IMU-IMU刚体约束消除对辅助惯性位姿的需求，从而降低计算复杂度。通过对MVIS的可观测性分析，我们证明了无论使用多少惯性传感器，标准的四个不可观测方向仍然存在，并首次识别了IMU-IMU时空外参和辅助惯性内参的退化运动。除了通过大量仿真验证我们的分析和算法外，我们还搭建了自己的MVIS传感器套件，并采集了超过25个真实世界数据集，以实验对比所提标定方法与最新标定方法（如Kalibr）的性能。结果表明，所提MVIS标定方法能够在收敛性和重复性方面取得具有竞争力的精度，该代码已开源以更好地惠及学术界。