Action Quality Assessment (AQA) is a task that tries to answer how well an action is carried out. While remarkable progress has been achieved, existing works on AQA assume that all the training data are visible for training in one time, but do not enable continual learning on assessing new technical actions. In this work, we address such a Continual Learning problem in AQA (Continual-AQA), which urges a unified model to learn AQA tasks sequentially without forgetting. Our idea for modeling Continual-AQA is to sequentially learn a task-consistent score-discriminative feature distribution, in which the latent features express a strong correlation with the score labels regardless of the task or action types. From this perspective, we aim to mitigate the forgetting in Continual-AQA from two aspects. Firstly, to fuse the features of new and previous data into a score-discriminative distribution, a novel Feature-Score Correlation-Aware Rehearsal is proposed to store and reuse data from previous tasks with limited memory size. Secondly, an Action General-Specific Graph is developed to learn and decouple the action-general and action-specific knowledge so that the task-consistent score-discriminative features can be better extracted across various tasks. Extensive experiments are conducted to evaluate the contributions of proposed components. The comparisons with the existing continual learning methods additionally verify the effectiveness and versatility of our approach.

翻译：动作质量评价（AQA）是一项旨在评估动作执行好坏的任务。尽管已有显著进展，现有AQA方法均假设所有训练数据可一次性用于训练，无法对新技术动作进行持续学习。本文针对AQA中的持续学习问题（Continual-AQA）展开研究，其核心需求是建立一个能够在不遗忘的前提下顺序学习AQA任务的统一模型。我们通过顺序学习任务一致的分数判别特征分布来建模Continual-AQA，其中潜在特征与分数标签呈现强相关性，且不受任务或动作类型影响。基于此，我们从两方面缓解Continual-AQA中的遗忘问题：首先，为将新旧数据特征融合为分数判别分布，提出一种特征-分数相关性感知回放机制，通过有限内存存储并复用先前任务数据；其次，构建动作泛化-特异性图，以学习并解耦动作泛化知识与特异性知识，从而跨不同任务更好地提取任务一致的分数判别特征。大量实验验证了所提组件的贡献，与现有持续学习方法的对比进一步证明了本方法的有效性与通用性。