Recent vision transformer based video models mostly follow the ``image pre-training then finetuning" paradigm and have achieved great success on multiple video benchmarks. However, full finetuning such a video model could be computationally expensive and unnecessary, given the pre-trained image transformer models have demonstrated exceptional transferability. In this work, we propose a novel method to Adapt pre-trained Image Models (AIM) for efficient video understanding. By freezing the pre-trained image model and adding a few lightweight Adapters, we introduce spatial adaptation, temporal adaptation and joint adaptation to gradually equip an image model with spatiotemporal reasoning capability. We show that our proposed AIM can achieve competitive or even better performance than prior arts with substantially fewer tunable parameters on four video action recognition benchmarks. Thanks to its simplicity, our method is also generally applicable to different image pre-trained models, which has the potential to leverage more powerful image foundation models in the future. The project webpage is \url{https://adapt-image-models.github.io/}.

翻译：近期基于视觉Transformer的视频模型大多遵循“图像预训练再微调”的范式，并在多个视频基准上取得了巨大成功。然而，考虑到预训练的图像Transformer模型已展现出卓越的可迁移性，对此类视频模型进行完全微调可能计算成本高昂且不必要。在这项工作中，我们提出一种新颖方法——适应预训练图像模型（AIM），以实现高效的视频理解。通过冻结预训练图像模型并添加少量轻量级适配器，我们引入了空间适应、时间适应和联合适应，逐步赋予图像模型时空推理能力。我们证明，所提出的AIM在四个视频动作识别基准上，以显著更少的可调参数即可达到与先前先进方法相当甚至更优的性能。由于方法的简洁性，其同样普遍适用于不同的图像预训练模型，这为未来利用更强大的图像基础模型提供了潜力。项目网页：\url{https://adapt-image-models.github.io/}。