Adapting large pretrained models to new tasks efficiently and continually is crucial for real-world deployment but remains challenging due to catastrophic forgetting and the high cost of retraining. While parameter-efficient tuning methods like low rank adaptation (LoRA) reduce computational demands, they lack mechanisms for strict continual learning and knowledge integration, without relying on data replay, or multiple adapters. We propose Share, a novel approach to parameter efficient continual finetuning that learns and dynamically updates a single, shared low-rank subspace, enabling seamless adaptation across multiple tasks and modalities. Share constructs a foundational subspace that extracts core knowledge from past tasks and incrementally integrates new information by identifying essential subspace directions. Knowledge from each new task is incorporated into this evolving subspace, facilitating forward knowledge transfer, while minimizing catastrophic interference. This approach achieves up to 100x parameter reduction and 281x memory savings over traditional LoRA methods, maintaining performance comparable to jointly trained models. A single Share model can replace hundreds of task-specific LoRA adapters, supporting scalable, asynchronous continual learning. Experiments across image classification, natural language understanding, 3D pose estimation, and text-to-image generation validate its effectiveness, making Share a practical and scalable solution for lifelong learning in large-scale AI systems.

翻译：将大型预训练模型高效且持续地适应新任务对于实际部署至关重要，但由于灾难性遗忘和重新训练的高成本，这仍然具有挑战性。虽然像低秩适应（LoRA）这样的参数高效调优方法降低了计算需求，但它们缺乏严格的持续学习和知识整合机制，且不依赖于数据回放或多个适配器。我们提出了Share，一种新颖的参数高效持续微调方法，它学习并动态更新一个单一的、共享的低秩子空间，从而能够在多个任务和模态之间实现无缝适应。Share构建了一个基础子空间，该子空间从过去任务中提取核心知识，并通过识别关键的子空间方向来增量整合新信息。每个新任务的知识都被纳入这个不断演化的子空间中，促进前向知识迁移，同时最小化灾难性干扰。与传统LoRA方法相比，该方法实现了高达100倍的参数减少和281倍的内存节省，同时保持了与联合训练模型相当的性能。一个单一的Share模型可以替代数百个特定于任务的LoRA适配器，支持可扩展的、异步的持续学习。在图像分类、自然语言理解、3D姿态估计和文本到图像生成等任务上的实验验证了其有效性，使Share成为大规模AI系统中终身学习的实用且可扩展的解决方案。