Methods for finetuning generative models for concept-driven personalization generally achieve strong results for subject-driven or style-driven generation. Recently, low-rank adaptations (LoRA) have been proposed as a parameter-efficient way of achieving concept-driven personalization. While recent work explores the combination of separate LoRAs to achieve joint generation of learned styles and subjects, existing techniques do not reliably address the problem; they often compromise either subject fidelity or style fidelity. We propose ZipLoRA, a method to cheaply and effectively merge independently trained style and subject LoRAs in order to achieve generation of any user-provided subject in any user-provided style. Experiments on a wide range of subject and style combinations show that ZipLoRA can generate compelling results with meaningful improvements over baselines in subject and style fidelity while preserving the ability to recontextualize. Project page: https://ziplora.github.io

翻译：针对概念驱动个性化生成模型的微调方法，通常在主题驱动或风格驱动生成任务中表现优异。近期，低秩适配（LoRA）被提出作为实现概念驱动个性化的一种参数高效方法。尽管已有工作探索通过组合独立LoRA实现学习风格与主题的联合生成，现有技术仍无法可靠解决该问题——它们常在主题保真度或风格保真度上有所妥协。我们提出ZipLoRA，一种经济高效合并独立训练的Style LoRA与Subject LoRA的方法，以实现用户提供的任意主题与任意风格的联合生成。在广泛主题与风格组合上的实验表明，ZipLoRA能在保持重新语境化能力的同时，生成令人信服的结果，并在主题与风格保真度上较基线方法取得显著改进。项目页面：https://ziplora.github.io