This paper proposes a vision-based framework for a 7-degree-of-freedom robotic manipulator, with the primary objective of facilitating its capacity to acquire information from human hand demonstrations for the execution of dexterous pick-and-place tasks. Most existing works only focus on the position demonstration without considering the orientations. In this paper, by employing a single depth camera, MediaPipe is applied to generate the three-dimensional coordinates of a human hand, thereby comprehensively recording the hand's motion, encompassing the trajectory of the wrist, orientation of the hand, and the grasp motion. A mean filter is applied during data pre-processing to smooth the raw data. The demonstration is designed to pick up an object at a specific angle, navigate around obstacles in its path and subsequently, deposit it within a sloped container. The robotic system demonstrates its learning capabilities, facilitated by the implementation of Dynamic Movement Primitives, enabling the assimilation of user actions into its trajectories with different start and end poi

翻译：本文提出了一种基于视觉的七自由度机器人操作臂框架，主要目标是赋予其从人类手部演示中获取信息以执行灵巧抓取-放置任务的能力。现有研究多聚焦于位置演示而未考虑姿态信息。本文通过单个深度相机，应用MediaPipe生成人体手部三维坐标，从而完整记录手部运动，包括腕部轨迹、手部姿态及抓取动作。数据预处理阶段采用均值滤波器对原始数据进行平滑处理。演示设计以特定角度抓取物体，沿路径规避障碍物，最终将其放入倾斜容器中。该机器人系统通过动态运动基元的实现展现了其学习能力，能够将用户动作融入其轨迹中，且适用于不同起始点和终止点。