Dexterous telemanipulation critically relies on the continuous and stable tracking of the human operator's commands to ensure robust operation. Vison-based tracking methods are widely used but have low stability due to anomalies such as occlusions, inadequate lighting, and loss of sight. Traditional filtering, regression, and interpolation methods are commonly used to compensate for explicit information such as angles and positions. These approaches are restricted to low-dimensional data and often result in information loss compared to the original high-dimensional image and video data. Recent advances in diffusion-based approaches, which can operate on high-dimensional data, have achieved remarkable success in video reconstruction and generation. However, these methods have not been fully explored in continuous control tasks in robotics. This work introduces the Diffusion-Enhanced Telemanipulation (DET) framework, which incorporates the Frame-Difference Detection (FDD) technique to identify and segment anomalies in video streams. These anomalous clips are replaced after reconstruction using diffusion models, ensuring robust telemanipulation performance under challenging visual conditions. We validated this approach in various anomaly scenarios and compared it with the baseline methods. Experiments show that DET achieves an average RMSE reduction of 17.2% compared to the cubic spline and 51.1% compared to FFT-based interpolation for different occlusion durations.

翻译：灵巧遥操作技术高度依赖于对操作员指令的连续稳定跟踪以确保鲁棒运行。基于视觉的跟踪方法虽被广泛采用，却因遮挡、光照不足及视野丢失等异常状况而稳定性较低。传统滤波、回归与插值方法通常用于补偿角度、位置等显式信息，但这些方法仅限于低维数据处理，相比原始高维图像与视频数据常导致信息损失。基于扩散模型的方法近期在高维数据处理方面取得显著进展，在视频重建与生成领域获得突出成果，然而这些方法在机器人连续控制任务中尚未得到充分探索。本研究提出扩散增强遥操作框架，该框架结合帧差检测技术以识别并分割视频流中的异常片段，并利用扩散模型对异常片段进行重建替换，从而在挑战性视觉条件下保障遥操作的鲁棒性能。我们在多种异常场景中验证了该方法，并与基线方法进行对比。实验表明：针对不同遮挡时长，DET框架相较于三次样条插值平均降低均方根误差17.2%，相较于基于快速傅里叶变换的插值方法平均降低51.1%。