Human-robot co-manipulation of soft materials, such as fabrics, composites, and sheets of paper/cardboard, is a challenging operation that presents several relevant industrial applications. Estimating the deformation state of the co-manipulated material is one of the main challenges. Viable methods provide the indirect measure by calculating the human-robot relative distance. In this paper, we develop a data-driven model to estimate the deformation state of the material from a depth image through a Convolutional Neural Network (CNN). First, we define the deformation state of the material as the relative roto-translation from the current robot pose and a human grasping position. The model estimates the current deformation state through a Convolutional Neural Network, specifically a DenseNet-121 pretrained on ImageNet.The delta between the current and the desired deformation state is fed to the robot controller that outputs twist commands. The paper describes the developed approach to acquire, preprocess the dataset and train the model. The model is compared with the current state-of-the-art method based on a skeletal tracker from cameras. Results show that our approach achieves better performances and avoids the various drawbacks caused by using a skeletal tracker.Finally, we also studied the model performance according to different architectures and dataset dimensions to minimize the time required for dataset acquisition

翻译：人机共操作软材料（如织物、复合材料、纸张/纸板）是一项具有挑战性的任务，涉及多项相关工业应用。估计共操作材料的变形状态是主要难点之一。可行的方法通过计算人机相对距离来提供间接测量。本文提出一种基于数据驱动的模型，通过卷积神经网络从深度图像估计材料变形状态。首先，将材料变形状态定义为当前机器人位姿与人类抓取位置之间的相对旋转平移量。模型通过卷积神经网络（具体为在ImageNet上预训练的DenseNet-121）估计当前变形状态。当前状态与期望变形状态之间的差值输入机器人控制器，输出扭转指令。本文阐述了数据采集、预处理及模型训练的具体方法，并与基于相机骨骼追踪器的现有最优方法进行对比。结果表明，我们的方法性能更优，且避免了骨骼追踪器带来的诸多缺陷。最后，我们进一步研究了不同架构和数据集规模对模型性能的影响，以缩短数据采集所需时间。