For privacy and security concerns, the need to erase unwanted information from pre-trained vision models is becoming evident nowadays. In real-world scenarios, erasure requests originate at any time from both users and model owners. These requests usually form a sequence. Therefore, under such a setting, selective information is expected to be continuously removed from a pre-trained model while maintaining the rest. We define this problem as continual forgetting and identify two key challenges. (i) For unwanted knowledge, efficient and effective deleting is crucial. (ii) For remaining knowledge, the impact brought by the forgetting procedure should be minimal. To address them, we propose Group Sparse LoRA (GS-LoRA). Specifically, towards (i), we use LoRA modules to fine-tune the FFN layers in Transformer blocks for each forgetting task independently, and towards (ii), a simple group sparse regularization is adopted, enabling automatic selection of specific LoRA groups and zeroing out the others. GS-LoRA is effective, parameter-efficient, data-efficient, and easy to implement. We conduct extensive experiments on face recognition, object detection and image classification and demonstrate that GS-LoRA manages to forget specific classes with minimal impact on other classes. Codes will be released on \url{https://github.com/bjzhb666/GS-LoRA}.

翻译：出于隐私和安全考虑，从预训练视觉模型中擦除不必要信息的需求正日益凸显。在实际场景中，用户和模型所有者随时可能提出擦除请求，这些请求通常形成序列。因此，在此设定下，需要从预训练模型中持续移除选择性信息，同时保留其余部分。我们将这一问题定义为持续遗忘，并识别出两个关键挑战：（i）对于不必要知识，需实现高效且有效的删除；（ii）对于保留知识，遗忘过程带来的影响应尽可能小。为解决这些问题，我们提出了群稀疏LoRA（GS-LoRA）。具体而言，针对（i），我们为每个遗忘任务独立使用LoRA模块微调Transformer块中的前馈神经网络层；针对（ii），采用简单的群稀疏正则化，自动选择特定LoRA群组并将其他群组归零。GS-LoRA具有高效、参数高效、数据高效且易于实现的特点。我们在人脸识别、目标检测和图像分类上进行了广泛实验，证明GS-LoRA能够在最小化对其他类别影响的同时遗忘特定类别。代码将在\url{https://github.com/bjzhb666/GS-LoRA}发布。