Egocentric RGB-D videos offer a natural source of human dexterous manipulation demonstrations, but existing data is difficult to use for robot learning because object pose, geometry, and contact information are often missing or require pre-scanned object assets. We present EgoAERO, the first framework that learns dexterous manipulation from a single egocentric RGB-D human demonstration without object assets. EgoAERO reconstructs contact-consistent hand-object trajectories through asset-free object tracking and reconstruction, ego motion compensation, and adaptive contact optimization, then converts them into robot policies using two-stage residual learning. We further introduce an online quality assessment mechanism and construct EgoDex-R, a large-scale egocentric dataset with 4.3M RGB-D frames for dexterous policy learning. Simulation and real-world experiments show that EgoAERO enables single-demonstration dexterous manipulation and achieves downstream performance close to CAD-based reconstructions on HOI4D.

翻译：第一人称RGB-D视频为人类灵巧操作示教提供了天然数据源，但现有数据难以直接用于机器人学习，原因在于物体姿态、几何形状及接触信息常缺失或需依赖预扫描的物体资产。本文提出EgoAERO——首个无需物体资产、仅凭单视角RGB-D人体示教视频即可学习灵巧操作的方法框架。EgoAERO通过无资产物体跟踪与重建、自运动补偿及自适应接触优化重建接触一致的手-物轨迹，进而采用两阶段残差学习将其转化为机器人策略。我们进一步引入在线质量评估机制，并构建EgoDex-R大规模第一人称数据集（包含430万帧RGB-D数据），用于灵巧策略学习。仿真与实物实验表明，EgoAERO能够实现单次示教下的灵巧操作，并在HOI4D基准上取得与基于CAD重建方法相媲美的下游性能。