Pneumatic soft robots present many advantages in manipulation tasks. Notably, their inherent compliance makes them safe and reliable in unstructured and fragile environments. However, full-body shape sensing for pneumatic soft robots is challenging because of their high degrees of freedom and complex deformation behaviors. Vision-based proprioception sensing methods relying on embedded cameras and deep learning provide a good solution to proprioception sensing by extracting the full-body shape information from the high-dimensional sensing data. But the current training data collection process makes it difficult for many applications. To address this challenge, we propose and demonstrate a robust sim-to-real pipeline that allows the collection of the soft robot's shape information in high-fidelity point cloud representation. The model trained on simulated data was evaluated with real internal camera images. The results show that the model performed with averaged Chamfer distance of 8.85 mm and tip position error of 10.12 mm even with external perturbation for a pneumatic soft robot with a length of 100.0 mm. We also demonstrated the sim-to-real pipeline's potential for exploring different configurations of visual patterns to improve vision-based reconstruction results. The code and dataset are available at https://github.com/DeepSoRo/DeepSoRoSim2Real.

翻译：气动软体机器人在操作任务中展现出诸多优势。特别是其固有的柔顺性使其在非结构化及脆弱环境中安全可靠。然而，由于气动软体机器人具有高自由度和复杂形变行为，其全身形状感知充满挑战。基于嵌入摄像头与深度学习的视觉本体感知方法通过从高维传感数据中提取全身形状信息，提供了良好的本体感知解决方案。但当前训练数据采集过程使其难以适用于许多应用场景。为应对这一挑战，我们提出并验证了一种鲁棒的模拟到现实迁移流水线，该流水线能够以高保真点云表示形式采集软体机器人的形状信息。在模拟数据上训练的模型通过真实内部摄像头图像进行评估。结果表明，即使存在外部扰动，该模型对于长度为100.0毫米的气动软体机器人仍能达到平均倒角距离8.85毫米和末端位置误差10.12毫米的性能。我们还展示了该模拟到现实流水线在探索视觉图案不同配置以改善视觉重建结果方面的潜力。代码与数据集已开源至https://github.com/DeepSoRo/DeepSoRoSim2Real。