In this paper, we study state estimation of multi-visual-inertial systems (MVIS) and develop sensor fusion algorithms to optimally fuse an arbitrary number of asynchronous inertial measurement units (IMUs) or gyroscopes and global and(or) rolling shutter cameras. We are especially interested in the full calibration of the associated visual-inertial sensors, including the IMU or camera intrinsics and the IMU-IMU(or camera) spatiotemporal extrinsics as well as the image readout time of rolling-shutter cameras (if used). To this end, we develop a new analytic combined IMU integration with intrinsics-termed ACI3-to preintegrate IMU measurements, which is leveraged to fuse auxiliary IMUs and(or) gyroscopes alongside a base IMU. We model the multi-inertial measurements to include all the necessary inertial intrinsic and IMU-IMU spatiotemporal extrinsic parameters, while leveraging IMU-IMU rigid-body constraints to eliminate the necessity of auxiliary inertial poses and thus reducing computational complexity. By performing observability analysis of MVIS, we prove that the standard four unobservable directions remain - no matter how many inertial sensors are used, and also identify, for the first time, degenerate motions for IMU-IMU spatiotemporal extrinsics and auxiliary inertial intrinsics. In addition to the extensive simulations that validate our analysis and algorithms, we have built our own MVIS sensor rig and collected over 25 real-world datasets to experimentally verify the proposed calibration against the state-of-the-art calibration method such as Kalibr. We show that the proposed MVIS calibration is able to achieve competing accuracy with improved convergence and repeatability, which is open sourced to better benefit the community.

翻译：本文研究了多视觉惯性系统（MVIS）的状态估计问题，并开发了传感器融合算法，以最优方式融合任意数量的异步惯性测量单元（IMU）或陀螺仪，以及全局快门和（或）卷帘快门相机。我们特别关注相关视觉惯性传感器的完整标定，包括IMU或相机内参、IMU-IMU（或相机）时空外参，以及卷帘快门相机的图像读出时间（若使用）。为此，我们开发了一种新的解析组合IMU积分方法（称为ACI3）对IMU测量进行预积分，该方法被用于在基础IMU之外融合辅助IMU和（或）陀螺仪。我们对多惯性测量进行建模，包含所有必要的惯性内参与IMU-IMU时空外参参数，同时利用IMU-IMU刚体约束消除对辅助惯性位姿的需求，从而降低计算复杂度。通过对MVIS进行可观测性分析，我们证明了无论使用多少个惯性传感器，标准的四个不可观测方向仍然存在，并首次识别出IMU-IMU时空外参与辅助惯性内参的退化运动。除了通过大量仿真验证我们的分析与算法外，我们还搭建了自研MVIS传感器平台，采集了超过25个真实世界数据集，以实验验证所提标定方法相较于当前最先进的标定方法（如Kalibr）的性能。结果表明，所提MVIS标定方法能够达到具有竞争力的精度，同时具有更好的收敛性与重复性，相关代码已开源以惠及学界。