In this paper, we tackle the problem of video alignment, the process of matching the frames of a pair of videos containing similar actions. The main challenge in video alignment is that accurate correspondence should be established despite the differences in the execution processes and appearances between the two videos. We introduce an unsupervised method for alignment that uses global and local features of the frames. In particular, we introduce effective features for each video frame by means of three machine vision tools: person detection, pose estimation, and VGG network. Then the features are processed and combined to construct a multidimensional time series that represent the video. The resulting time series are used to align videos of the same actions using a novel version of dynamic time warping named Diagonalized Dynamic Time Warping(DDTW). The main advantage of our approach is that no training is required, which makes it applicable for any new type of action without any need to collect training samples for it. For evaluation, we considered video synchronization and phase classification tasks on the Penn action dataset. Also, for an effective evaluation of the video synchronization task, we present a new metric called Enclosed Area Error(EAE). The results show that our method outperforms previous state-of-the-art methods, such as TCC and other self-supervised and supervised methods.

翻译：本文研究了视频对齐问题，即匹配包含相似动作的视频对的帧。视频对齐的主要挑战在于，尽管两个视频的执行过程和表观存在差异，仍需建立精确的对应关系。我们提出了一种无监督对齐方法，利用帧的全局和局部特征。具体而言，通过三种机器视觉工具（人物检测、姿态估计和VGG网络）为每一视频帧提取有效特征，然后对特征进行处理与融合，构建表示视频的多维时间序列。利用所得时间序列，采用名为对角化动态时间规整（DDTW）的新型动态时间规整算法，对相同动作的视频进行对齐。该方法的主要优势在于无需训练，因此可适用于任何新型动作，无需收集训练样本。在评估中，我们基于Penn动作数据集进行了视频同步与相位分类任务。此外，为有效评估视频同步任务，我们提出了一种名为包围面积误差（EAE）的新指标。结果表明，我们的方法优于先前最先进的方法，如TCC及其他自监督与监督方法。