Recently, dataset distillation has paved the way towards efficient machine learning, especially for image datasets. However, the distillation for videos, characterized by an exclusive temporal dimension, remains an underexplored domain. In this work, we provide the first systematic study of video distillation and introduce a taxonomy to categorize temporal compression. Our investigation reveals that the temporal information is usually not well learned during distillation , and the temporal dimension of synthetic data contributes little. The observations motivate our unified framework of disentangling the dynamic and static information in the videos. It first distills the videos into still images as static memory and then compensates the dynamic and motion information with a learnable dynamic memory block. Our method achieves state-of-the-art on video datasets at different scales, with notably smaller storage expenditure. Our code will be publicly available.

翻译：近年来，数据集蒸馏为高效机器学习铺平了道路，尤其是在图像数据集领域。然而，针对具有独特时间维度的视频蒸馏仍是一个尚未充分探索的领域。本研究首次系统性地研究视频蒸馏，并引入了一套分类体系来归纳时间压缩方法。我们的研究发现，在蒸馏过程中时间信息通常未能得到充分学习，且合成数据的时间维度贡献甚微。这一观察促使我们提出了一个解耦视频中动态与静态信息的统一框架：该方法首先将视频蒸馏为静态图像作为静态记忆，随后通过可学习的动态记忆块补偿动态与运动信息。我们的方法在不同规模的视频数据集上均取得了最先进的表现，且存储开销显著更小。我们的代码将开源。