In this work, a deep learning-based technique is used to study the image-to-joint inverse kinematics of a tendon-driven supportive continuum arm. An eye-off-hand configuration is considered by mounting a camera at a fixed pose with respect to the inertial frame attached at the arm base. This camera captures an image for each distinct joint variable at each sampling time to construct the training dataset. This dataset is then employed to adapt a feed-forward deep convolutional neural network, namely the modified VGG-16 model, to estimate the joint variable. One thousand images are recorded to train the deep network, and transfer learning and fine-tuning techniques are applied to the modified VGG-16 to further improve the training. Finally, training is also completed with a larger dataset of images that are affected by various types of noises, changes in illumination, and partial occlusion. The main contribution of this research is the development of an image-to-joint network that can estimate the joint variable given an image of the arm, even if the image is not captured in an ideal condition. The key benefits of this research are twofold: 1) image-to-joint mapping can offer a real-time alternative to computationally complex inverse kinematic mapping through analytical models; and 2) the proposed technique can provide robustness against noise, occlusion, and changes in illumination. The dataset is publicly available on Kaggle.

翻译：本研究采用基于深度学习的技术，研究肌腱驱动支撑型连续体臂的图像到关节逆运动学问题。通过将相机固定安装于与臂基座惯性坐标系相对位姿不变的位置，构建了眼在手上的配置方案。该相机在每个采样时刻针对不同关节变量采集图像，以构建训练数据集。随后利用该数据集对前馈深度卷积神经网络（即改进的VGG-16模型）进行适配，以实现关节变量的估计。研究记录了千幅图像用于深度网络训练，并对改进的VGG-16模型应用迁移学习与微调技术以进一步提升训练效果。最终，训练还采用了包含多种噪声干扰、光照变化及部分遮挡情况的大型图像数据集。本研究的主要贡献在于开发了一种图像到关节网络，该网络能够根据机械臂图像估计关节变量，即使在非理想拍摄条件下仍能保持性能。本研究的核心优势体现在两方面：1）相较于通过解析模型实现计算复杂度较高的逆运动学映射，图像到关节映射可提供实时替代方案；2）所提技术对噪声、遮挡及光照变化具有鲁棒性。相关数据集已在Kaggle平台公开。