This paper proposes a vision-based framework for a 7-degree-of-freedom robotic manipulator, with the primary objective of facilitating its capacity to acquire information from human hand demonstrations for the execution of dexterous pick-and-place tasks. Most existing works only focus on the position demonstration without considering the orientations. In this paper, by employing a single depth camera, MediaPipe is applied to generate the three-dimensional coordinates of a human hand, thereby comprehensively recording the hand's motion, encompassing the trajectory of the wrist, orientation of the hand, and the grasp motion. A mean filter is applied during data pre-processing to smooth the raw data. The demonstration is designed to pick up an object at a specific angle, navigate around obstacles in its path and subsequently, deposit it within a sloped container. The robotic system demonstrates its learning capabilities, facilitated by the implementation of Dynamic Movement Primitives, enabling the assimilation of user actions into its trajectories with different start and end poi

翻译：本文提出了一种基于视觉的七自由度机械臂操作框架，其主要目标是使机械臂能够从人手演示中获取信息，以执行灵巧的抓取-放置任务。现有研究大多仅关注位置演示，而未考虑姿态信息。本文通过使用单目深度相机，应用MediaPipe生成人手的三维坐标，从而全面记录手部运动，包括手腕轨迹、手部姿态以及抓握动作。在数据预处理阶段，采用均值滤波器对原始数据进行平滑处理。演示设计为以特定角度抓取物体，绕过路径中的障碍物，随后将其放入倾斜容器中。该机器人系统通过动态运动基元的实现，展示了其学习能力，能够将用户动作融入具有不同起始点和终点的轨迹中。