In many real-world settings, image observations of freely rotating 3D rigid bodies, such as satellites, may be available when low-dimensional measurements are not. However, the high-dimensionality of image data precludes the use of classical estimation techniques to learn the dynamics and a lack of interpretability reduces the usefulness of standard deep learning methods. In this work, we present a physics-informed neural network model to estimate and predict 3D rotational dynamics from image sequences. We achieve this using a multi-stage prediction pipeline that maps individual images to a latent representation homeomorphic to $\mathbf{SO}(3)$, computes angular velocities from latent pairs, and predicts future latent states using the Hamiltonian equations of motion with a learned representation of the Hamiltonian. We demonstrate the efficacy of our approach on a new rotating rigid-body dataset with sequences of rotating cubes and rectangular prisms with uniform and non-uniform density.

翻译：在许多实际场景中，当无法获取低维测量数据时，可能可以获得自由旋转3D刚体（如卫星）的图像观测数据。然而，图像数据的高维性阻碍了经典估计技术用于动力学学习，而可解释性的缺失又降低了标准深度学习方法的应用价值。本文提出了一种物理信息神经网络模型，用于从图像序列中估计和预测三维旋转动力学。我们通过多阶段预测流水线实现该目标：将单幅图像映射至与$\mathbf{SO}(3)$同胚的潜空间表征，从潜空间表征对中计算角速度，并利用哈密顿运动方程及学习到的哈密顿量表征预测未来潜状态。我们在新构建的旋转刚体数据集上验证了方法的有效性，该数据集包含旋转立方体和矩形棱柱（均匀和非均匀密度）的图像序列。