State estimation for legged robots is challenging due to their highly dynamic motion and limitations imposed by sensor accuracy. By integrating Kalman filtering, optimization, and learning-based modalities, we propose a hybrid solution that combines proprioception and exteroceptive information for estimating the state of the robot's trunk. Leveraging joint encoder and IMU measurements, our Kalman filter is enhanced through a single-rigid body model that incorporates ground reaction force control outputs from convex Model Predictive Control optimization. The estimation is further refined through Gated Recurrent Units, which also considers semantic insights and robot height from a Vision Transformer autoencoder applied on depth images. This framework not only furnishes accurate robot state estimates, including uncertainty evaluations, but can minimize the nonlinear errors that arise from sensor measurements and model simplifications through learning. The proposed methodology is evaluated in hardware using a quadruped robot on various terrains, yielding a 65% improvement on the Root Mean Squared Error compared to our VIO SLAM baseline. Code example: https://github.com/AlexS28/OptiState

翻译：腿式机器人的状态估计因其高度动态运动及传感器精度限制而具有挑战性。通过整合卡尔曼滤波、优化与学习方法，我们提出一种融合本体感觉与外感受信息的混合方案，用于估计机器人躯干状态。利用关节编码器和惯性测量单元（IMU）数据，我们的卡尔曼滤波器通过引入包含凸模型预测控制（MPC）优化生成的地面反作用力控制输出的单刚体模型得到增强。估计过程进一步通过门控循环单元（GRU）精化，同时结合来自深度图像的视觉Transformer自编码器提取的语义信息与机器人高度。该框架不仅提供包含不确定性评估的精确机器人状态估计，还能通过学习最小化由传感器测量和模型简化导致的非线性误差。所提方法在四足机器人的多种地形硬件实验中进行了评估，相较于我们的VIO SLAM基线，均方根误差（RMSE）提升了65%。代码示例：https://github.com/AlexS28/OptiState