Recent advances in the field of intelligent robotic manipulation pursue providing robotic hands with touch sensitivity. Haptic perception encompasses the sensing modalities encountered in the sense of touch (e.g., tactile and kinesthetic sensations). This letter focuses on multimodal object recognition and proposes analytical and data-driven methodologies to fuse tactile- and kinesthetic-based classification results. The procedure is as follows: a three-finger actuated gripper with an integrated high-resolution tactile sensor performs squeeze-and-release Exploratory Procedures (EPs). The tactile images and kinesthetic information acquired using angular sensors on the finger joints constitute the time-series datasets of interest. Each temporal dataset is fed to a Long Short-term Memory (LSTM) Neural Network, which is trained to classify in-hand objects. The LSTMs provide an estimation of the posterior probability of each object given the corresponding measurements, which after fusion allows to estimate the object through Bayesian and Neural inference approaches. An experiment with 36-classes is carried out to evaluate and compare the performance of the fused, tactile, and kinesthetic perception systems.The results show that the Bayesian-based classifiers improves capabilities for object recognition and outperforms the Neural-based approach.

翻译：近年来，智能机器人操作领域的进展致力于为机械手赋予触觉感知能力。触觉感知涵盖触觉与动觉等多种感觉模态。本文聚焦于多模态物体识别问题，提出基于分析与数据驱动的方法融合触觉与动觉分类结果。具体流程如下：使用集成高分辨率触觉传感器的三指驱动夹爪执行挤压释放式探索程序。通过手指关节角度传感器获取的触觉图像与动觉信息构成感兴趣的时序数据集。每个时序数据被输入至长短时记忆神经网络，该网络经训练用于分类手持物体。LSTM提供基于对应测量的各物体后验概率估计，融合后可通过贝叶斯与神经推理方法估计物体类别。通过包含36类物体的实验评估并比较融合系统、触觉系统及动觉系统的性能。结果表明，基于贝叶斯的分类器显著提升了物体识别能力，且其性能优于基于神经推理的方法。