Liquids and granular media are pervasive throughout human environments. Their free-flowing nature causes people to constrain them into containers. We do so with thousands of different types of containers made out of different materials with varying sizes, shapes, and colors. In this work, we present a state-of-the-art sensing technique for robots to perceive what liquid is inside of an unknown container. We do so by integrating Visible to Near Infrared (VNIR) reflectance spectroscopy into a robot's end effector. We introduce a hierarchical model for inferring the material classes of both containers and internal contents given spectral measurements from two integrated spectrometers. To train these inference models, we capture and open source a dataset of spectral measurements from over 180 different combinations of containers and liquids. Our technique demonstrates over 85% accuracy in identifying 13 different liquids and granular media contained within 13 different containers. The sensitivity of our spectral readings allow our model to also identify the material composition of the containers themselves with 96% accuracy. Overall, VNIR spectroscopy presents a promising method to give household robots a general-purpose ability to infer the liquids inside of containers, without needing to open or manipulate the containers.

翻译：液体和颗粒介质在人类环境中无处不在。其自由流动的特性促使人们将其约束于容器中。我们使用的容器种类超过数千种，由不同材料制成，尺寸、形状和颜色各异。本研究提出一种先进的机器人传感技术，使机器人能够感知未知容器内的液体。该技术通过将可见光至近红外反射光谱集成到机器人的末端执行器中实现。我们引入了一个分层模型，利用两个集成光谱仪获取的光谱测量数据，同时推断容器和内部容物的材料类别。为训练这些推断模型，我们采集并开源了一个包含180余种容器与液体组合的光谱测量数据集。该技术在识别13种不同容器中的13种液体及颗粒介质时，准确率超过85%。光谱读数的灵敏度使我们的模型还能以96%的准确率识别容器本身的材料成分。总体而言，VNIR光谱学为赋予家用机器人无需开盖或操作容器即可推断容器内液体的通用能力，提供了一种极具前景的方法。