State estimation from measured data is crucial for robotic applications as autonomous systems rely on sensors to capture the motion and localize in the 3D world. Among sensors that are designed for measuring a robot's pose, or for soft robots, their shape, vision sensors are favorable because they are information-rich, easy to set up, and cost-effective. With recent advancements in computer vision, deep learning-based methods no longer require markers for identifying feature points on the robot. However, learning-based methods are data-hungry and hence not suitable for soft and prototyping robots, as building such bench-marking datasets is usually infeasible. In this work, we achieve image-based robot pose estimation and shape reconstruction from camera images. Our method requires no precise robot meshes, but rather utilizes a differentiable renderer and primitive shapes. It hence can be applied to robots for which CAD models might not be available or are crude. Our parameter estimation pipeline is fully differentiable. The robot shape and pose are estimated iteratively by back-propagating the image loss to update the parameters. We demonstrate that our method of using geometrical shape primitives can achieve high accuracy in shape reconstruction for a soft continuum robot and pose estimation for a robot manipulator.

翻译：从测量数据中进行状态估计对机器人应用至关重要，因为自主系统需要依靠传感器来捕捉运动并在三维世界中定位。在用于测量机器人姿态（或对软体机器人而言测量其形状）的传感器中，视觉传感器因其信息丰富、易于设置且成本效益高而备受青睐。随着计算机视觉的最新进展，基于深度学习的方法不再需要标记物来识别机器人上的特征点。然而，基于学习的方法对数据需求量大，因而不适用于软体机器人和原型机器人，因为构建此类基准数据集通常不可行。在这项工作中，我们实现了基于相机图像的机器人姿态估计与形状重建。我们的方法无需精确的机器人网格模型，而是利用可微渲染器和基本几何形状。因此，该方法适用于那些可能没有CAD模型或CAD模型粗糙的机器人。我们的参数估计流程是完全可微的。通过反向传播图像损失来更新参数，从而迭代估计机器人的形状和姿态。我们证明了使用几何形状基元的方法能够在对软体连续型机器人进行形状重建以及对机器人操作臂进行姿态估计时实现高精度。