Simulating optical tactile sensors presents significant challenges due to their high deformability and intricate optical properties. To address these issues and enable a physically accurate simulation, we propose DOT-Sim: Differentiable Optical Tactile Simulation. Unlike prior simulators that rely on simplified models of deformable sensors, DOT-Sim accurately captures the physical behavior of soft sensors by modeling them as elastic materials using the Material Point Method (MPM). DOT-Sim enables rapid calibration of optical tactile sensor simulation using a small number of demonstrations within minutes, which is substantially faster than existing methods. Compared to current baselines, our approach supports much larger and non-linear deformations. To handle the optical aspect, we propose a novel approach to simulating optical responses by learning a residual image relative to the real-world idle state. We validate the physical and visual realism of our method through a series of zero-shot sim-to-real tasks. Our experiments show that DOT-Sim (1) accurately replicates the physical dynamics of a DenseTact optical tactile sensor in reality, (2) generates realistic optical outputs in contact-rich scenarios, (3) enables direct deployment of simulation-trained classifiers in the real world, achieving 85% classification accuracy on challenging objects and 90% accuracy in embedded tumor-type detection, and (4) allows precise trajectory following with a policy trained from demonstrations in simulation, with an average error of less than 0.9 mm.

翻译：摘要：由于光学触觉传感器具有高变形性和复杂的光学特性，对其进行仿真面临重大挑战。为解决这些问题并实现物理精确的仿真，我们提出DOT-Sim：可微分光学触觉仿真。与依赖简化可变形传感器模型的先前仿真器不同，DOT-Sim通过使用物质点法（MPM）将软传感器建模为弹性材料，从而准确捕捉其物理行为。DOT-Sim能在数分钟内利用少量演示数据快速校准光学触觉传感器仿真，速度显著快于现有方法。与当前基线相比，我们的方法支持更大范围且非线性的形变。针对光学方面，我们提出一种新型方法，通过学习与真实世界空闲状态相关的残差图像来仿真光学响应。我们通过一系列零样本仿到实任务验证了该方法的物理和视觉真实性。实验表明，DOT-Sim能够：（1）准确复现真实DenseTact光学触觉传感器的物理动力学行为；（2）在密集接触场景中生成逼真的光学输出；（3）实现仿真训练分类器在真实世界的直接部署，在挑战性物体分类任务中达到85%的准确率，在嵌入式肿瘤类型检测中达到90%的准确率；（4）支持基于仿真演示训练的策略实现精确轨迹跟踪，平均误差低于0.9毫米。