We introduce SonicSense, a holistic design of hardware and software to enable rich robot object perception through in-hand acoustic vibration sensing. While previous studies have shown promising results with acoustic sensing for object perception, current solutions are constrained to a handful of objects with simple geometries and homogeneous materials, single-finger sensing, and mixing training and testing on the same objects. SonicSense enables container inventory status differentiation, heterogeneous material prediction, 3D shape reconstruction, and object re-identification from a diverse set of 83 real-world objects. Our system employs a simple but effective heuristic exploration policy to interact with the objects as well as end-to-end learning-based algorithms to fuse vibration signals to infer object properties. Our framework underscores the significance of in-hand acoustic vibration sensing in advancing robot tactile perception.

翻译：我们提出了SonicSense，一种硬件与软件协同设计的整体方案，通过手持声学振动感知实现丰富的机器人物体感知能力。尽管先前研究已展示声学传感在物体感知方面的潜力，但现有方案通常局限于几何形状简单、材料均匀的少数物体，采用单指传感模式，且在相同物体上进行训练与测试。SonicSense能够对83种真实世界物体实现容器存量状态区分、异质材料预测、三维形状重建及物体重识别。我们的系统采用简洁而高效的启发式探索策略与物体交互，并利用端到端学习算法融合振动信号以推断物体属性。该框架凸显了手持声学振动传感在推进机器人触觉感知领域的重要价值。