We present a parameter-efficient method for continual video question-answering (VidQA) learning. Our method, named DAM, uses the proposed Dynamic Adapter Merging to (i) mitigate catastrophic forgetting, (ii) enable efficient adaptation to continually arriving datasets, (iii) handle inputs from unknown datasets during inference, and (iv) enable knowledge sharing across similar dataset domains. Given a set of continually streaming VidQA datasets, we sequentially train dataset-specific adapters for each dataset while freezing the parameters of a large pretrained video-language backbone. During inference, given a video-question sample from an unknown domain, our method first uses the proposed non-parametric router function to compute a probability for each adapter, reflecting how relevant that adapter is to the current video-question input instance. Subsequently, the proposed dynamic adapter merging scheme aggregates all the adapter weights into a new adapter instance tailored for that particular test sample to compute the final VidQA prediction, mitigating the impact of inaccurate router predictions and facilitating knowledge sharing across domains. Our DAM model outperforms prior state-of-the-art continual learning approaches by 9.1% while exhibiting 1.9% less forgetting on 6 VidQA datasets spanning various domains. We further extend DAM to continual image classification and image QA and outperform prior methods by a large margin. The code is publicly available at: https://github.com/klauscc/DAM

翻译：我们提出了一种用于连续视频问答（VidQA）学习的参数高效方法。该方法名为DAM，采用所提出的动态适配器融合技术，旨在：(i) 缓解灾难性遗忘，(ii) 实现对持续到达数据集的高效适应，(iii) 在推理阶段处理来自未知数据集的输入，以及(iv) 实现跨相似数据集领域的知识共享。给定一组连续流式VidQA数据集，我们在冻结大型预训练视频-语言骨干网络参数的同时，依次为每个数据集训练专用的适配器。推理时，面对来自未知领域的视频-问题样本，我们的方法首先利用所提出的非参数路由函数计算每个适配器的概率，以反映该适配器与当前视频-问题输入实例的相关性。随后，动态适配器融合方案将所有适配器权重聚合为一个针对该特定测试样本定制的新适配器实例，用于计算最终的VidQA预测，从而缓解路由预测不准确的影响并促进跨领域知识共享。我们的DAM模型在涵盖多个领域的6个VidQA数据集上，相较于先前最先进的持续学习方法实现了9.1%的性能提升，同时遗忘率降低了1.9%。我们进一步将DAM扩展至持续图像分类和图像问答任务，并以较大幅度超越了现有方法。代码已开源于：https://github.com/klauscc/DAM