Action Quality Assessment (AQA) is a task that tries to answer how well an action is carried out. While remarkable progress has been achieved, existing works on AQA assume that all the training data are visible for training at one time, but do not enable continual learning on assessing new technical actions. In this work, we address such a Continual Learning problem in AQA (Continual-AQA), which urges a unified model to learn AQA tasks sequentially without forgetting. Our idea for modeling Continual-AQA is to sequentially learn a task-consistent score-discriminative feature distribution, in which the latent features express a strong correlation with the score labels regardless of the task or action types.From this perspective, we aim to mitigate the forgetting in Continual-AQA from two aspects. Firstly, to fuse the features of new and previous data into a score-discriminative distribution, a novel Feature-Score Correlation-Aware Rehearsal is proposed to store and reuse data from previous tasks with limited memory size. Secondly, an Action General-Specific Graph is developed to learn and decouple the action-general and action-specific knowledge so that the task-consistent score-discriminative features can be better extracted across various tasks. Extensive experiments are conducted to evaluate the contributions of proposed components. The comparisons with the existing continual learning methods additionally verify the effectiveness and versatility of our approach. Data and code are available at https://github.com/iSEE-Laboratory/Continual-AQA.

翻译：动作质量评估（AQA）是一项旨在回答动作执行效果如何的任务。尽管已取得显著进展，现有AQA研究均假设所有训练数据可一次性用于训练，且未能实现对新技术动作的持续学习评估。本文针对AQA中的持续学习问题（Continual-AQA）展开研究，该问题要求统一模型在不遗忘旧知识的前提下顺序学习AQA任务。我们的建模思路是顺序学习一种任务一致的分数判别性特征分布，其中潜在特征与分数标签呈现强相关性，不受任务或动作类型影响。基于此视角，我们拟从两方面缓解Continual-AQA中的遗忘问题：首先，为将新旧数据特征融合为分数判别性分布，提出一种新颖的"特征-分数相关性感知重放"方法，在有限内存条件下存储并复现旧任务数据；其次，开发"动作通用-专用图"以学习并解耦动作通用知识与动作专用知识，从而跨不同任务更优提取任务一致的分数判别性特征。通过大量实验验证了各组件的贡献。与现有持续学习方法的对比进一步证明了我们方法的有效性与通用性。数据和代码已开源：https://github.com/iSEE-Laboratory/Continual-AQA。