Constructing photorealistic and controllable robotic arm digital assets from real observations is fundamental to robotic applications. Current approaches naively bind static 3D Gaussians according to URDF links, forcing them to follow an URDF-rigged motion passively. However, the idealized URDF-rigged motion cannot accurately model the actual motion captured in real-world observations, leading to severe rendering artifacts in 3D Gaussians. To address these challenges, we propose RoboArmGS, a novel hybrid representation that refines the URDF-rigged motion with learnable Bézier curves, enabling more accurate real-world motion modeling. To be more specific, we present a learnable Bézier Curve motion refiner that corrects per-joint residuals to address mismatches between real-world motion and URDF-rigged motion. RoboArmGS enables the learning of more accurate real-world motion while achieving a coherent binding of 3D Gaussians across arm parts. To support future research, we contribute a carefully collected dataset named RoboArm4D, which comprises several widely used robotic arms for evaluating the quality of building high-quality digital assets. We evaluate our approach on RoboArm4D, and RoboArmGS achieves state-of-the-art performance in real-world motion modeling and rendering quality. The code and dataset will be released.
翻译:从真实观测构建具有照片级真实感且可控的机械臂数字资产是机器人应用的基础。现有方法通常简单地将静态3D高斯根据URDF连杆进行绑定,迫使其被动遵循URDF骨架驱动的运动。然而,理想化的URDF骨架运动无法准确建模真实世界观测中捕获的实际运动,导致3D高斯产生严重的渲染伪影。为解决这些挑战,我们提出RoboArmGS——一种通过可学习贝塞尔曲线优化URDF骨架运动的新型混合表示方法,能够实现更精确的真实世界运动建模。具体而言,我们设计了一个可学习的贝塞尔曲线运动优化器,通过校正每个关节的残差来解决真实运动与URDF骨架运动之间的失配问题。RoboArmGS在实现跨机械臂部件3D高斯连贯绑定的同时,能够学习更精确的真实世界运动。为支持未来研究,我们贡献了精心收集的数据集RoboArm4D,其中包含多个广泛使用的机械臂,用于评估高质量数字资产的构建质量。我们在RoboArm4D上评估了所提方法,RoboArmGS在真实运动建模与渲染质量方面均达到了最先进的性能。代码与数据集将公开。