Pose based video sychronization can have applications in multiple domains such as gameplay performance evaluation, choreography or guiding athletes. The subject's actions could be compared and evaluated against those performed by professionals side by side. In this paper, we propose an end to end pipeline for synchronizing videos based on pose. The first step crops the region where the person present in the image followed by pose detection on the cropped image. This is followed by application of Dynamic Time Warping(DTW) on angle/ distance measures between the pose keypoints leading to a scale and shift invariant pose matching pipeline.
翻译:基于姿态的视频同步可应用于多个领域,如游戏行为评估、编舞或运动员指导。受试者的动作可与专业人士的动作进行并排比较和评估。本文提出了一种基于姿态端到端的视频同步流程。第一步裁剪图像中人物所在的区域,随后对裁剪后的图像进行姿态检测。接着,将动态时间规整(DTW)应用于姿态关键点之间的角度/距离度量,从而形成一种尺度与平移不变的姿态匹配流程。