We present OmniVIC, a universal variable impedance controller (VIC) enhanced by a vision language model (VLM), which improves safety and adaptation in any contact-rich robotic manipulation task to enhance safe physical interaction. Traditional VIC have shown advantages when the robot physically interacts with the environment, but lack generalization in unseen, complex, and unstructured safe interactions in universal task scenarios involving contact or uncertainty. To this end, the proposed OmniVIC interprets task context derived reasoning from images and natural language and generates adaptive impedance parameters for a VIC controller. Specifically, the core of OmniVIC is a self-improving Retrieval-Augmented Generation(RAG) and in-context learning (ICL), where RAG retrieves relevant prior experiences from a structured memory bank to inform the controller about similar past tasks, and ICL leverages these retrieved examples and the prompt of current task to query the VLM for generating context-aware and adaptive impedance parameters for the current manipulation scenario. Therefore, a self-improved RAG and ICL guarantee OmniVIC works in universal task scenarios. The impedance parameter regulation is further informed by real-time force/torque feedback to ensure interaction forces remain within safe thresholds. We demonstrate that our method outperforms baselines on a suite of complex contact-rich tasks, both in simulation and on real-world robotic tasks, with improved success rates and reduced force violations. OmniVIC takes a step towards bridging high-level semantic reasoning and low-level compliant control, enabling safer and more generalizable manipulation. Overall, the average success rate increases from 27% (baseline) to 61.4% (OmniVIC).

翻译：本文提出OmniVIC，一种由视觉语言模型增强的通用可变阻抗控制器，旨在提升任意接触密集型机器人操作任务中的安全性与适应性，从而增强安全的物理交互能力。传统可变阻抗控制器在机器人与环境进行物理交互时已展现出优势，但在涉及接触或不确定性的通用任务场景中，面对未见过的、复杂的、非结构化的安全交互时缺乏泛化能力。为此，所提出的OmniVIC能够从图像和自然语言中解析任务上下文推理，并为可变阻抗控制器生成自适应阻抗参数。具体而言，OmniVIC的核心是一个自优化的检索增强生成与上下文学习框架：其中检索增强生成从结构化记忆库中检索相关的先验经验，使控制器了解过往类似任务；上下文学习则利用这些检索到的示例及当前任务的提示信息查询视觉语言模型，从而为当前操作场景生成具有上下文感知能力的自适应阻抗参数。因此，自优化的检索增强生成与上下文学习机制保证了OmniVIC在通用任务场景中的适用性。阻抗参数调节还进一步结合实时力/力矩反馈，以确保交互力始终维持在安全阈值内。我们通过仿真和真实机器人任务验证了本方法在一系列复杂接触密集型任务上均优于基线方法，在提升成功率的同时降低了力违规情况。OmniVIC朝着弥合高层语义推理与底层顺应控制的方向迈进了一步，实现了更安全、泛化能力更强的操作能力。总体而言，平均成功率从基线方法的27%提升至OmniVIC的61.4%。