Our paper proposes a direct sparse visual odometry method that combines event and RGB-D data to estimate the pose of agile-legged robots during dynamic locomotion and acrobatic behaviors. Event cameras offer high temporal resolution and dynamic range, which can eliminate the issue of blurred RGB images during fast movements. This unique strength holds a potential for accurate pose estimation of agile-legged robots, which has been a challenging problem to tackle. Our framework leverages the benefits of both RGB-D and event cameras to achieve robust and accurate pose estimation, even during dynamic maneuvers such as jumping and landing a quadruped robot, the Mini-Cheetah. Our major contributions are threefold: Firstly, we introduce an adaptive time surface (ATS) method that addresses the whiteout and blackout issue in conventional time surfaces by formulating pixel-wise decay rates based on scene complexity and motion speed. Secondly, we develop an effective pixel selection method that directly samples from event data and applies sample filtering through ATS, enabling us to pick pixels on distinct features. Lastly, we propose a nonlinear pose optimization formula that simultaneously performs 3D-2D alignment on both RGB-based and event-based maps and images, allowing the algorithm to fully exploit the benefits of both data streams. We extensively evaluate the performance of our framework on both public datasets and our own quadruped robot dataset, demonstrating its effectiveness in accurately estimating the pose of agile robots during dynamic movements.

翻译：本文提出一种直接稀疏视觉里程计方法，结合事件数据与RGB-D数据，用于估计敏捷足式机器人在动态运动及杂技行为中的位姿。事件相机具有高时间分辨率和高动态范围的优势，能够消除快速运动时RGB图像的模糊问题。这一独特特性为准确估计敏捷足式机器人位姿这一难题提供了潜力。我们的框架充分利用RGB-D相机与事件相机的优势，即使在四足机器人Mini-Cheetah执行跳跃、着陆等动态动作时，也能实现鲁棒且准确的位姿估计。本文的主要贡献有三方面：首先，我们提出自适应时间表面（ATS）方法，通过基于场景复杂度和运动速度逐像素计算衰减率，解决了传统时间表面的白化与黑化问题。其次，我们开发了一种高效像素选择方法，直接从事件数据中采样，并通过ATS进行样本过滤，从而选取具有显著特征的像素。最后，我们提出一种非线性位姿优化公式，可同时对基于RGB和基于事件的图像与地图进行三维-二维对齐，使算法能够充分利用两种数据流的优势。我们通过在公开数据集及自建四足机器人数据集上的全面评估，证明了该方法在动态运动中准确估计敏捷机器人位姿的有效性。