Employing a teleoperation system for gathering demonstrations offers the potential for more efficient learning of robot manipulation. However, teleoperating a robot arm equipped with a dexterous hand or gripper, via a teleoperation system poses significant challenges due to its high dimensionality, complex motions, and differences in physiological structure. In this study, we introduce a novel system for joint learning between human operators and robots, that enables human operators to share control of a robot end-effector with a learned assistive agent, facilitating simultaneous human demonstration collection and robot manipulation teaching. In this setup, as data accumulates, the assistive agent gradually learns. Consequently, less human effort and attention are required, enhancing the efficiency of the data collection process. It also allows the human operator to adjust the control ratio to achieve a trade-off between manual and automated control. We conducted experiments in both simulated environments and physical real-world settings. Through user studies and quantitative evaluations, it is evident that the proposed system could enhance data collection efficiency and reduce the need for human adaptation while ensuring the collected data is of sufficient quality for downstream tasks. Videos are available at https://norweig1an.github.io/human-agent-joint-learning.github.io/.

翻译：利用遥操作系统收集演示数据，有望实现更高效的机器人操作学习。然而，通过遥操作系统操控配备灵巧手或抓取器的机器人手臂，因其高维度、复杂运动及生理结构差异而面临重大挑战。本研究提出一种人机协同学习的新型系统，使人类操作者能够与习得的辅助智能体共享机器人末端执行器的控制权，从而同步实现人类演示数据收集与机器人操作教学。在此框架下，随着数据不断积累，辅助智能体逐步学习。因此，所需的人类操作精力与注意力随之减少，显著提升了数据收集过程的效率。该系统还允许人类操作者调节控制比例，实现手动控制与自动控制之间的权衡。我们在仿真环境与物理现实场景中均进行了实验。通过用户研究与定量评估，结果表明所提出的系统能够提升数据收集效率，减少对人类适应性的依赖，同时确保所收集数据对下游任务具有足够质量。演示视频详见 https://norweig1an.github.io/human-agent-joint-learning.github.io/。