Humans learn about objects via interaction and using multiple perceptions, such as vision, sound, and touch. While vision can provide information about an object's appearance, non-visual sensors, such as audio and haptics, can provide information about its intrinsic properties, such as weight, temperature, hardness, and the object's sound. Using tools to interact with objects can reveal additional object properties that are otherwise hidden (e.g., knives and spoons can be used to examine the properties of food, including its texture and consistency). Robots can use tools to interact with objects and gather information about their implicit properties via non-visual sensors. However, a robot's model for recognizing objects using a tool-mediated behavior does not generalize to a new tool or behavior due to differing observed data distributions. To address this challenge, we propose a framework to enable robots to transfer implicit knowledge about granular objects across different tools and behaviors. The proposed approach learns a shared latent space from multiple robots' contexts produced by respective sensory data while interacting with objects using tools. We collected a dataset using a UR5 robot that performed 5,400 interactions using 6 tools and 6 behaviors on 15 granular objects and tested our method on cross-tool and cross-behavioral transfer tasks. Our results show the less experienced target robot can benefit from the experience gained from the source robot and perform recognition on a set of novel objects. We have released the code, datasets, and additional results: https://github.com/gtatiya/Tool-Knowledge-Transfer.

翻译：人类通过互动和多种感知（如视觉、声音和触觉）来学习物体知识。视觉能提供物体外观信息，而音频与触觉等非视觉传感器则可揭示物体的内在属性，如重量、温度、硬度及物体发出的声音。使用工具与物体互动能够进一步发现其他方式难以获取的物体属性（例如，用刀和勺子可检测食物的质地与稠度）。机器人可利用工具与物体互动，并通过非视觉传感器收集其隐含属性信息。然而，机器人基于工具介导行为构建的物体识别模型，由于观测数据分布差异，难以推广至新工具或新行为。为解决这一挑战，我们提出一个框架，使机器人能够跨不同工具和行为迁移关于颗粒状物体的隐含知识。该方法通过多个机器人在使用工具与物体互动时产生的各自感知数据，学习一个共享潜在空间。我们使用UR5机器人采集了一个数据集，该机器人使用6种工具和6种行为对15种颗粒状物体执行了5400次互动，并基于跨工具与跨行为迁移任务测试了方法。结果表明，经验较少的目标机器人可从源机器人的经验中获益，并对一组新颖物体进行识别。我们已开源代码、数据集及补充结果：https://github.com/gtatiya/Tool-Knowledge-Transfer。