The fine-grained action analysis of the existing action datasets is challenged by insufficient action categories, low fine granularities, limited modalities, and tasks. In this paper, we propose a Multi-modality and Multi-task dataset of Figure Skating (MMFS) which was collected from the World Figure Skating Championships. MMFS, which possesses action recognition and action quality assessment, captures RGB, skeleton, and is collected the score of actions from 11671 clips with 256 categories including spatial and temporal labels. The key contributions of our dataset fall into three aspects as follows. (1) Independently spatial and temporal categories are first proposed to further explore fine-grained action recognition and quality assessment. (2) MMFS first introduces the skeleton modality for complex fine-grained action quality assessment. (3) Our multi-modality and multi-task dataset encourage more action analysis models. To benchmark our dataset, we adopt RGB-based and skeleton-based baseline methods for action recognition and action quality assessment.

翻译：现有动作数据集的细粒度分析面临动作类别不足、细粒度低、模态有限以及任务类型单一等挑战。本文提出一个多模态多任务的花样滑冰数据集（MMFS），该数据集收集自世界花样滑冰锦标赛。MMFS支持动作识别与动作质量评估，包含RGB图像、骨骼数据，并收录了来自11671个视频片段中256个类别的动作得分，同时提供空间与时间标签。本数据集的主要贡献体现在以下三个方面：（1）首次独立提出空间与时间类别，以进一步探索细粒度动作识别与质量评估；（2）MMFS首次引入骨骼模态用于复杂细粒度动作质量评估；（3）我们的多模态多任务数据集可促进更多动作分析模型的发展。为建立基准评估，我们采用基于RGB和基于骨骼的基线方法进行动作识别与动作质量评估。