In this work, a deep learning-based technique is used to study the image-to-joint inverse kinematics of a tendon-driven supportive continuum arm. An eye-off-hand configuration is considered by mounting a camera at a fixed pose with respect to the inertial frame attached at the arm base. This camera captures an image for each distinct joint variable at each sampling time to construct the training dataset. This dataset is then employed to adapt a feed-forward deep convolutional neural network, namely the modified VGG-16 model, to estimate the joint variable. One thousand images are recorded to train the deep network, and transfer learning and fine-tuning techniques are applied to the modified VGG-16 to further improve the training. Finally, training is also completed with a larger dataset of images that are affected by various types of noises, changes in illumination, and partial occlusion. The main contribution of this research is the development of an image-to-joint network that can estimate the joint variable given an image of the arm, even if the image is not captured in an ideal condition. The key benefits of this research are twofold: 1) image-to-joint mapping can offer a real-time alternative to computationally complex inverse kinematic mapping through analytical models; and 2) the proposed technique can provide robustness against noise, occlusion, and changes in illumination. The dataset is publicly available on Kaggle.

翻译：本研究采用基于深度学习的技术，研究肌腱驱动支撑型连续体臂的图像到关节逆运动学问题。通过将相机固定安装于与臂基座惯性坐标系相对位姿不变的位置，构建了眼在手外的配置方案。该相机在每个采样时刻针对不同关节变量采集图像，以此构建训练数据集。该数据集随后用于调整前馈深度卷积神经网络（即改进的VGG-16模型）以估计关节变量。研究采集了一千张图像训练深度网络，并对改进的VGG-16模型应用迁移学习与微调技术以进一步提升训练效果。最终，训练还采用了包含多种噪声干扰、光照变化及部分遮挡情况的大型图像数据集完成。本研究的主要贡献在于开发了一种图像到关节网络，能够根据机械臂图像估计关节变量，即使图像并非在理想条件下采集。本研究的核心优势体现在两方面：1）相较于通过解析模型实现计算复杂度高的逆运动学映射，图像到关节映射可提供实时替代方案；2）所提技术对噪声、遮挡及光照变化具有鲁棒性。数据集已在Kaggle平台公开。