Open-source foundation models have seen rapid adoption and development, enabling powerful general-purpose capabilities across diverse domains. However, fine-tuning large foundation models for domain-specific or personalized tasks remains prohibitively expensive for most users due to the significant memory overhead beyond that of inference. We introduce EMLoC, an Emulator-based Memory-efficient fine-tuning framework with LoRA Correction, which enables model fine-tuning within the same memory budget required for inference. EMLoC constructs a task-specific light-weight emulator using activation-aware singular value decomposition (SVD) on a small downstream calibration set. Fine-tuning then is performed on this lightweight emulator via LoRA. To tackle the misalignment between the original model and the compressed emulator, we propose a novel compensation algorithm to correct the fine-tuned LoRA module, which thus can be merged into the original model for inference. EMLoC supports flexible compression ratios and standard training pipelines, making it adaptable to a wide range of applications. Extensive experiments demonstrate that EMLoC outperforms other baselines across multiple datasets and modalities. Moreover, without quantization, EMLoC enables fine-tuning of a 38B model, which originally required 95GB of memory, on a single 24GB consumer GPU-bringing efficient and practical model adaptation to individual users.

翻译：开源基础模型已获得快速采用与发展，使其能够在多样化领域中展现出强大的通用能力。然而，针对特定领域或个性化任务对大型基础模型进行微调，由于所需内存开销远超推理阶段，对大多数用户而言仍然成本高昂。我们提出了EMLoC，一种基于模拟器的内存高效微调框架，具备LoRA校正功能，该框架能够在与推理相同的内存预算内实现模型微调。EMLoC利用下游小型校准集，通过激活感知奇异值分解（SVD）构建一个任务特定的轻量级模拟器。随后通过LoRA在此轻量级模拟器上进行微调。为解决原始模型与压缩模拟器之间的不对齐问题，我们提出了一种新颖的补偿算法来校正微调后的LoRA模块，从而使其能够合并到原始模型中以供推理使用。EMLoC支持灵活的压缩比和标准训练流程，使其能够适应广泛的应用场景。大量实验表明，EMLoC在多个数据集和模态上均优于其他基线方法。此外，在不进行量化的前提下，EMLoC使得原本需要95GB内存的380亿参数模型能够在单块24GB消费级GPU上完成微调，从而为个人用户带来了高效且实用的模型适配能力。