Data scarcity has long been an issue in the robot learning community. Particularly, in safety-critical domains like surgical applications, obtaining high-quality data can be especially difficult. It poses challenges to researchers seeking to exploit recent advancements in reinforcement learning and imitation learning, which have greatly improved generalizability and enabled robots to conduct tasks autonomously. We introduce dARt Vinci, a scalable data collection platform for robot learning in surgical settings. The system uses Augmented Reality (AR) hand tracking and a high-fidelity physics engine to capture subtle maneuvers in primitive surgical tasks: By eliminating the need for a physical robot setup and providing flexibility in terms of time, space, and hardware resources-such as multiview sensors and actuators-specialized simulation is a viable alternative. At the same time, AR allows the robot data collection to be more egocentric, supported by its body tracking and content overlaying capabilities. Our user study confirms the proposed system's efficiency and usability, where we use widely-used primitive tasks for training teleoperation with da Vinci surgical robots. Data throughput improves across all tasks compared to real robot settings by 41% on average. The total experiment time is reduced by an average of 10%. The temporal demand in the task load survey is improved. These gains are statistically significant. Additionally, the collected data is over 400 times smaller in size, requiring far less storage while achieving double the frequency.

翻译：数据稀缺长期以来一直是机器人学习领域面临的问题。特别是在手术应用等安全关键领域，获取高质量数据尤为困难。这给希望利用强化学习和模仿学习最新进展的研究者带来了挑战，这些进展极大地提升了泛化能力并使机器人能够自主执行任务。我们提出了dARt Vinci，一个用于手术场景下机器人学习的可扩展数据采集平台。该系统利用增强现实（AR）手部追踪和高保真物理引擎，捕捉基础外科任务中的精细操作：通过消除对实体机器人装置的需求，并在时间、空间及硬件资源（如多视角传感器与执行器）方面提供灵活性，专用仿真成为一种可行的替代方案。同时，AR凭借其身体追踪与内容叠加能力，使机器人数据采集更具自我中心特性。我们的用户研究证实了所提系统的效率与可用性，其中我们采用广泛使用的基础任务来训练达芬奇手术机器人的遥操作。与真实机器人环境相比，所有任务的数据吞吐量平均提升41%，总实验时间平均减少10%，任务负荷调查中的时间需求指标得到改善。这些增益均具有统计显著性。此外，采集的数据规模缩小超过400倍，在存储需求大幅降低的同时实现了双倍的采集频率。