Handheld paradigms offer an efficient and intuitive way for collecting large-scale demonstration of robot manipulation. However, achieving contact-rich bimanual manipulation through these methods remains a pivotal challenge, which is substantially hindered by hardware adaptability and data efficacy. Prior hardware designs remain gripper-specific and often face a trade-off between tracking precision and portability. Furthermore, the lack of online feasibility checking during demonstration leads to poor replayability. More importantly, existing handheld setups struggle to collect interactive recovery data during robot execution, lacking the authentic tactile information necessary for robust policy refinement. To bridge these gaps, we present TAMEn, a tactile-aware manipulation engine for closed-loop data collection in contact-rich tasks. Our system features a cross-morphology wearable interface that enables rapid adaptation across heterogeneous grippers. To balance data quality and environmental diversity, we implement a dual-modal acquisition pipeline: a precision mode leveraging motion capture for high-fidelity demonstrations, and a portable mode utilizing VR-based tracking for in-the-wild acquisition and tactile-visualized recovery teleoperation. Building on this hardware, we unify large-scale tactile pretraining, task-specific bimanual demonstrations, and human-in-the-loop recovery data into a pyramid-structured data regime, enabling closed-loop policy refinement. Experiments show that our feasibility-aware pipeline significantly improves demonstration replayability, and that the proposed visuo-tactile learning framework increases task success rates from 34% to 75% across diverse bimanual manipulation tasks. We further open-source the hardware and dataset to facilitate reproducibility and support research in visuo-tactile manipulation.
翻译:手持式范式为大规模机器人操控演示数据采集提供了高效直观的途径。然而,通过此类方法实现接触密集的双臂操控仍面临关键挑战,硬件适应性与数据效能构成主要障碍。现有硬件设计局限于特定夹爪,且在跟踪精度与便携性之间往往存在取舍。此外,演示过程中缺乏在线可行性校验导致数据回放性能低下。更重要的是,现有手持式设备难以在机器人执行阶段采集交互式恢复数据,缺乏鲁棒策略优化所需的真实触觉信息。为弥合这些不足,我们提出TAMEn——面向接触密集任务的闭环数据采集触觉感知操控引擎。本系统采用跨形态可穿戴接口,能快速适配异构夹爪。为平衡数据质量与环境多样性,我们实现双模态采集流水线:基于动作捕捉的高保真演示精度模式,以及基于VR跟踪的野外采集与触觉可视化恢复遥操作便携模式。基于该硬件体系,我们将大规模触觉预训练、任务特异性双臂演示和人在回路恢复数据整合为金字塔结构数据机制,实现闭环策略优化。实验表明,具备可行性感知能力的流水线显著提升演示回放性能,所提出的视觉-触觉学习框架将多样化双臂操控任务的成功率从34%提升至75%。我们进一步开源硬件与数据集,以促进可复现性并支持视觉-触觉操控研究。