Soft pneumatic robot manipulators are popular in industrial and human-interactive applications due to their compliance and flexibility. However, deploying them in real-world scenarios requires advanced sensing for tactile feedback and proprioception. Our work presents a novel vision-based approach for sensorizing soft robots. We demonstrate our approach on PneuGelSight, a pioneering pneumatic manipulator featuring high-resolution proprioception and tactile sensing via an embedded camera. To optimize the sensor's performance, we introduce a comprehensive pipeline that accurately simulates its optical and dynamic properties, facilitating a zero-shot knowledge transition from simulation to real-world applications. PneuGelSight and our sim-to-real pipeline provide a novel, easily implementable, and robust sensing methodology for soft robots, paving the way for the development of more advanced soft robots with enhanced sensory capabilities.

翻译：软体气动机器人机械臂因其顺应性与灵活性，在工业及人机交互应用中备受青睐。然而，要将其部署于实际场景，需要先进的传感技术以实现触觉反馈与本体感知。本研究提出了一种基于视觉的新型方法，用于为软体机器人赋予感知能力。我们在PneuGelSight上验证了该方法，这是一种开创性的气动机械臂，通过嵌入式摄像头实现了高分辨率的本体感知与触觉传感。为优化传感器性能，我们引入了一套完整的流程，能够精确模拟其光学与动态特性，从而促进从仿真到实际应用的零样本知识迁移。PneuGelSight及我们的仿真到现实流程为软体机器人提供了一种新颖、易于实现且鲁棒的传感方法，为开发具有更强感知能力的先进软体机器人铺平了道路。