Multi-perspective cameras with potentially non-overlapping fields of view have become an important exteroceptive sensing modality in a number of applications such as intelligent vehicles, drones, and mixed reality headsets. In this work, we challenge one of the basic assumptions made in these scenarios, which is that the multi-camera rig is rigid. More specifically, we are considering the problem of estimating the relative pose between a static non-rigid rig in different spatial orientations while taking into account the effect of gravity onto the system. The deformable physical connections between each camera and the body center are approximated by a simple cantilever model, and inserted into the generalized epipolar constraint. Our results lead us to the important insight that the latent parameters of the deformation model, meaning the gravity vector in both views, become observable. We present a concise analysis of the observability of all variables based on noise, outliers, and rig rigidity for two different algorithms. The first one is a vision-only alternative, while the second one makes use of additional gravity measurements. To conclude, we demonstrate the ability to sense gravity in a real-world example, and discuss practical implications.

翻译：具有潜在非重叠视场的多视角相机已成为智能车辆、无人机及混合现实头显等应用中的重要外部感知模态。本文挑战了此类场景中的一个基本假设——多相机系统为刚性结构。具体而言，我们研究了在考虑重力对系统影响的前提下，静态非刚性相机系统在不同空间朝向下的相对位姿估计问题。通过简化的悬臂梁模型近似各相机与身体中心之间的可变形物理连接，并将其引入广义对极约束。研究结果揭示了一个重要发现：变形模型的潜在参数（即两个视角中的重力向量）变得可观测。针对两种不同算法，我们基于噪声、外点和系统刚性对所有变量的可观测性进行了简明分析：第一种为纯视觉方案，第二种则利用额外的重力测量值。最后，我们通过真实世界实例展示了感知重力的能力，并讨论了实际应用意义。