The increasing demand for sustainable textile recycling requires robust automation solutions capable of handling deformable garments and detecting foreign objects in cluttered environments. This work presents a digital twin driven robotic sorting system that integrates grasp prediction, multi modal perception, and semantic reasoning for real world textile classification. A dual arm robotic cell equipped with RGBD sensing, capacitive tactile feedback, and collision-aware motion planning autonomously separates garments from an unsorted basket, transfers them to an inspection zone, and classifies them using state of the art Visual Language Models (VLMs). We benchmark nine VLM s from five model families on a dataset of 223 inspection scenarios comprising shirts, socks, trousers, underwear, foreign objects (including garments outside of the aforementioned classes), and empty scenes. The evaluation assesses per class accuracy, hallucination behavior, and computational performance under practical hardware constraints. Results show that the Qwen model family achieves the highest overall accuracy (up to 87.9 %), with strong foreign object detection performance, while lighter models such as Gemma3 offer competitive speed accuracy trade offs for edge deployment. A digital twin combined with MoveIt enables collision aware path planning and integrates segmented 3D point clouds of inspected garments into the virtual environment for improved manipulation reliability. The presented system demonstrates the feasibility of combining semantic VLM reasoning with conventional grasp detection and digital twin technology for scalable, autonomous textile sorting in realistic industrial settings.
翻译:日益增长的可持续纺织品回收需求要求开发能够处理可变形衣物并在杂乱环境中检测异物的稳健自动化解决方案。本文提出了一种数字孪生驱动的机器人物料分拣系统,该系统结合了抓取预测、多模态感知与语义推理,用于真实场景下的纺织品分类。配备RGBD传感、电容式触觉反馈及碰撞感知运动规划的双臂机器人单元,可从散料筐中自主分离衣物,将其转移至检测区域,并利用最先进的视觉语言模型(VLMs)进行分类。我们在包含衬衫、袜子、裤子、内衣、异物(包括上述类别以外的衣物)及空场景共223个检测场景的数据集上,对来自五个模型家族的九个VLM进行了基准测试。评估涵盖了各类别准确率、幻觉行为以及在实际硬件约束下的计算性能。结果表明,Qwen模型家族取得了最高的总体准确率(高达87.9%),并具备优异的异物检测性能,而Gemma3等轻量级模型则为边缘部署提供了极具竞争力的速度-准确率权衡。通过结合数字孪生与MoveIt,系统实现了碰撞感知的路径规划,并将已检测衣物的分割三维点云集成至虚拟环境中,从而提升了操作可靠性。本文所提出的系统验证了将语义VLM推理与传统抓取检测及数字孪生技术相结合,在现实工业环境中实现可扩展的自主纺织品分拣的可行性。