Deformable objects often appear in unstructured configurations. Tracing deformable objects helps bringing them into extended states and facilitating the downstream manipulation tasks. Due to the requirements for object-specific modeling or sim-to-real transfer, existing tracing methods either lack generalizability across different categories of deformable objects or struggle to complete tasks reliably in the real world. To address this, we propose a novel visual-tactile imitation learning method to achieve one-dimensional (1D) and two-dimensional (2D) deformable object tracing with a unified model. Our method is designed from both local and global perspectives based on visual and tactile sensing. Locally, we introduce a weighted loss that emphasizes actions maintaining contact near the center of the tactile image, improving fine-grained adjustment. Globally, we propose a tracing task loss that helps the policy to regulate task progression. On the hardware side, to compensate for the limited features extracted from visual information, we integrate tactile sensing into a low-cost teleoperation system considering both the teleoperator and the robot. Extensive ablation and comparative experiments on diverse 1D and 2D deformable objects demonstrate the effectiveness of our approach, achieving an average success rate of 80% on seen objects and 65% on unseen objects.

翻译：可变形物体常以非结构化形态出现。追踪可变形物体有助于将其延展至伸展状态，从而便于后续操作任务。现有追踪方法因需要物体特定建模或虚实迁移，难以兼顾跨类别可变形物体的泛化能力与真实场景的可靠执行。为此，我们提出一种新颖的可视化-触觉模仿学习方法，通过统一模型实现一维（1D）与二维（2D）可变形物体追踪。该方法基于视觉与触觉感知，从局部与全局双视角设计：局部层面，引入加权损失函数强化触觉图像中心区域接触维持动作，提升精细调节能力；全局层面，提出追踪任务损失函数引导策略调控任务进程。硬件层面，为弥补视觉信息特征提取不足，我们将触觉感知集成至低成本的遥操作系统中（同时考虑遥操作设备与机器人可行性）。针对多种一维与二维可变形物的大量消融实验与对比实验表明，本方法对已知物体平均成功率可达80%，对未知物体达65%。