Human-robot co-manipulation of soft materials, such as fabrics, composites, and sheets of paper/cardboard, is a challenging operation that presents several relevant industrial applications. Estimating the deformation state of the co-manipulated material is one of the main challenges. Viable methods provide the indirect measure by calculating the human-robot relative distance. In this paper, we develop a data-driven model to estimate the deformation state of the material from a depth image through a Convolutional Neural Network (CNN). First, we define the deformation state of the material as the relative roto-translation from the current robot pose and a human grasping position. The model estimates the current deformation state through a Convolutional Neural Network, specifically a DenseNet-121 pretrained on ImageNet.The delta between the current and the desired deformation state is fed to the robot controller that outputs twist commands. The paper describes the developed approach to acquire, preprocess the dataset and train the model. The model is compared with the current state-of-the-art method based on a skeletal tracker from cameras. Results show that our approach achieves better performances and avoids the various drawbacks caused by using a skeletal tracker.Finally, we also studied the model performance according to different architectures and dataset dimensions to minimize the time required for dataset acquisition

翻译：人机共操作软材料（如织物、复合材料、纸张/纸板等）是一项具有挑战性的任务，涉及多种工业应用场景。其中，对共操作材料形变状态的估计是主要难点之一。现有可行方法通过计算人机相对距离来间接测量形变。本文提出一种数据驱动模型，利用卷积神经网络(CNN)从深度图像中估计材料形变状态。首先，将材料形变状态定义为当前机器人位姿与人类抓取点之间的相对旋转平移量。模型通过卷积神经网络（具体采用在ImageNet上预训练的DenseNet-121）估计当前形变状态。当前形变状态与目标形变状态之间的差值被输入机器人控制器，输出扭转指令。本文详细阐述了数据集采集、预处理及模型训练方法，并与基于摄像头骨骼跟踪器的当前最优方法进行对比。结果表明，本文方法性能更优且避免了骨骼跟踪器的多种缺陷。最后，本文还研究了不同网络架构和数据集规模对模型性能的影响，以最大程度减少数据集采集所需时间。