Knowledge of terrain's physical properties inferred from color images can aid in making efficient robotic locomotion plans. However, unlike image classification, it is unintuitive for humans to label image patches with physical properties. Without labeled data, building a vision system that takes as input the observed terrain and predicts physical properties remains challenging. We present a method that overcomes this challenge by self-supervised labeling of images captured by robots during real-world traversal with physical property estimators trained in simulation. To ensure accurate labeling, we introduce Active Sensing Motor Policies (ASMP), which are trained to explore locomotion behaviors that increase the accuracy of estimating physical parameters. For instance, the quadruped robot learns to swipe its foot against the ground to estimate the friction coefficient accurately. We show that the visual system trained with a small amount of real-world traversal data accurately predicts physical parameters. The trained system is robust and works even with overhead images captured by a drone despite being trained on data collected by cameras attached to a quadruped robot walking on the ground.

翻译：从彩色图像中推断地形的物理属性有助于制定高效的机器人运动规划。然而，与图像分类不同，人类难以直觉地为图像块标注物理属性。在没有标注数据的情况下，构建一个以观测地形为输入并预测物理属性的视觉系统仍然具有挑战性。我们提出了一种方法，通过利用在仿真中训练的物理属性估计器，对机器人在真实世界遍历过程中捕捉的图像进行自监督标注，从而克服了这一挑战。为确保标注的准确性，我们引入了主动感知运动策略（ASMP），这些策略经过训练，可以探索能够提高物理参数估计精度的运动行为。例如，四足机器人学会用脚摩擦地面以准确估计摩擦系数。我们展示了使用少量真实世界遍历数据训练的视觉系统能够准确预测物理参数。尽管训练数据是由地面行走的四足机器人上的相机收集的，但训练后的系统具有鲁棒性，甚至能够处理无人机从空中拍摄的图像。