Tactile servoing is an important technique because it enables robots to manipulate objects with precision and accuracy while adapting to changes in their environments in real-time. One approach for tactile servo control with high-resolution soft tactile sensors is to estimate the contact pose relative to an object surface using a convolutional neural network (CNN) for use as a feedback signal. In this paper, we investigate how the surface pose estimation model can be extended to include shear, and utilize these combined pose-and-shear models to develop a tactile robotic system that can be programmed for diverse non-prehensile manipulation tasks, such as object tracking, surface following, single-arm object pushing and dual-arm object pushing. In doing this, two technical challenges had to be overcome. Firstly, the use of tactile data that includes shear-induced slippage can lead to error-prone estimates unsuitable for accurate control, and so we modified the CNN into a Gaussian-density neural network and used a discriminative Bayesian filter to improve the predictions with a state dynamics model that utilizes the robot kinematics. Secondly, to achieve smooth robot motion in 3D space while interacting with objects, we used SE(3) velocity-based servo control, which required re-deriving the Bayesian filter update equations using Lie group theory, as many standard assumptions do not hold for state variables defined on non-Euclidean manifolds. In future, we believe that pose and shear-based tactile servoing will enable many object manipulation tasks and the fully-dexterous utilization of multi-fingered tactile robot hands. Video: https://www.youtube.com/watch?v=xVs4hd34ek0

翻译：触觉伺服控制是一项重要技术，它使机器人能够在实时适应环境变化的同时，以高精度完成物体操控。利用高分辨率软体触觉传感器实现触觉伺服控制的一种方法，是通过卷积神经网络(CNN)估计相对于物体表面的接触位姿，并将其作为反馈信号。本文研究了如何扩展表面位姿估计模型以纳入剪切力信息，并利用这种位姿-剪切力联合模型构建了一套可编程的触觉机器人系统，该系统能够完成多种非抓取操控任务（如目标跟踪、表面跟随、单臂物体推拉、双臂物体推拉）。在此过程中需克服两个技术难题：首先，包含剪切诱发滑移的触觉数据可能导致误差较大的估计，不适用于精确控制，因此我们将CNN改进为高斯密度神经网络，并采用判别式贝叶斯滤波器结合利用机器人运动学的状态动力学模型来提升预测精度；其次，为在三维空间中与物体交互时实现平滑的机器人运动，我们采用基于SE(3)速度的伺服控制，这需要利用李群理论重新推导贝叶斯滤波器更新方程——因为许多标准假设不适用于定义在非欧流形上的状态变量。未来，我们认为基于位姿与剪切力的触觉伺服控制将赋能多种物体操控任务，并实现多指触觉机器人手的全灵巧化应用。视频地址：https://www.youtube.com/watch?v=xVs4hd34ek0