Humans learn about objects via interaction and using multiple perceptions, such as vision, sound, and touch. While vision can provide information about an object's appearance, non-visual sensors, such as audio and haptics, can provide information about its intrinsic properties, such as weight, temperature, hardness, and the object's sound. Using tools to interact with objects can reveal additional object properties that are otherwise hidden (e.g., knives and spoons can be used to examine the properties of food, including its texture and consistency). Robots can use tools to interact with objects and gather information about their implicit properties via non-visual sensors. However, a robot's model for recognizing objects using a tool-mediated behavior does not generalize to a new tool or behavior due to differing observed data distributions. To address this challenge, we propose a framework to enable robots to transfer implicit knowledge about granular objects across different tools and behaviors. The proposed approach learns a shared latent space from multiple robots' contexts produced by respective sensory data while interacting with objects using tools. We collected a dataset using a UR5 robot that performed 5,400 interactions using 6 tools and 6 behaviors on 15 granular objects and tested our method on cross-tool and cross-behavioral transfer tasks. Our results show the less experienced target robot can benefit from the experience gained from the source robot and perform recognition on a set of novel objects. We have released the code, datasets, and additional results: https://github.com/gtatiya/Tool-Knowledge-Transfer.

翻译：人类通过交互与多重感知（如视觉、听觉、触觉）学习物体知识。视觉可提供物体外观信息，而音频、触觉等非视觉传感器则能揭示重量、温度、硬度等内在属性及物体发出的声响。使用工具与物体交互可进一步发现隐藏属性（例如，用刀和勺子可检测食物的质地与稠度）。机器人可借助工具通过非视觉传感器采集隐式属性信息。然而，由于观测数据分布存在差异，机器人基于工具中介行为建立的物体识别模型无法泛化至新工具或新行为。针对这一挑战，我们提出了一种框架，使机器人能够跨不同工具和行为迁移颗粒状物体的隐式知识。该方法通过多机器人使用工具交互时各自传感数据产生的上下文，构建共享隐空间。我们利用UR5机器人采集数据集，对15种颗粒状物体执行了5400次交互（涉及6种工具与6种行为），并在跨工具与跨行为迁移任务中验证了方法有效性。结果表明，经验较少的靶机器人可从源机器人的经验中获益，实现对未见物体的识别。相关代码、数据集及附加结果已开源：https://github.com/gtatiya/Tool-Knowledge-Transfer。