Point-supervised Temporal Action Localization (PTAL) adopts a lightly frame-annotated paradigm (\textit{i.e.}, labeling only a single frame per action instance) to train a model to effectively locate action instances within untrimmed videos. Most existing approaches design the task head of models with only a point-supervised snippet-level classification, without explicit modeling of understanding temporal relationships among frames of an action. However, understanding the temporal relationships of frames is crucial because it can help a model understand how an action is defined and therefore benefits localizing the full frames of an action. To this end, in this paper, we design a multi-task learning framework that fully utilizes point supervision to boost the model's temporal understanding capability for action localization. Specifically, we design three self-supervised temporal understanding tasks: (i) Action Completion, (ii) Action Order Understanding, and (iii) Action Regularity Understanding. These tasks help a model understand the temporal consistency of actions across videos. To the best of our knowledge, this is the first attempt to explicitly explore temporal consistency for point supervision action localization. Extensive experimental results on four benchmark datasets demonstrate the effectiveness of the proposed method compared to several state-of-the-art approaches.

翻译：点监督时序动作定位采用轻量级帧标注范式（即每个动作实例仅标注单帧）来训练模型，以在未修剪视频中有效定位动作实例。现有方法大多仅设计基于点监督的片段级分类任务头，未显式建模动作帧间的时序关系理解。然而，理解帧间时序关系至关重要，因其能帮助模型理解动作的定义方式，从而有利于定位动作的完整帧序列。为此，本文设计了一个多任务学习框架，充分利用点监督来增强模型的动作定位时序理解能力。具体而言，我们设计了三个自监督时序理解任务：（i）动作补全，（ii）动作顺序理解，以及（iii）动作规律性理解。这些任务帮助模型理解视频中动作的时序一致性。据我们所知，这是首次显式探索时序一致性用于点监督动作定位的尝试。在四个基准数据集上的大量实验结果表明，与多种先进方法相比，所提方法具有显著有效性。