Large Language Models (LLMs) have demonstrated remarkable efficacy in text embedding, yet current adaptation methods like LoRA face significant bottlenecks in computational efficiency and cross-architecture transferability. Whenever a new backbone emerges, existing approaches require costly retraining from scratch. To address this, we propose PromptEmbedder, a novel dual-LLM framework that decouples embedding knowledge from specific backbone weights. PromptEmbedder utilizes a Prompting LLM to generate instruction-aware soft prompts for a frozen Embedding LLM via a differentiable generation process with continuous relaxation, ensuring full gradient flow during contrastive training. By localizing task-specific knowledge within the Prompting LLM, adapting to new architectures requires only retraining a lightweight linear alignment matrix. Evaluations on the MTEB benchmark show that PromptEmbedder achieves comparable performance with LoRA finetuning while reducing GPU memory by 40% and accelerating training by 3.7x. Our approach establishes a scalable, architecture-agnostic paradigm for efficient LLM-based representation learning.

翻译：大语言模型（LLMs）在文本嵌入任务中展现出显著效果，但当前如LoRA等适配方法在计算效率和跨架构迁移性方面存在明显瓶颈。每当新骨干模型出现时，现有方法需要从头进行昂贵的重新训练。为此，我们提出PromptEmbedder——一种新型双LLM框架，将嵌入知识与特定骨干网络权重解耦。PromptEmbedder利用提示LLM通过连续松弛的可微分生成过程，为冻结的嵌入LLM生成指令感知的软提示，确保对比训练过程中梯度的完整流动。通过将任务特定知识定位在提示LLM中，适配新架构仅需重新训练轻量级线性对齐矩阵。在MTEB基准上的评估表明，PromptEmbedder在实现与LoRA微调相当性能的同时，将GPU内存消耗降低40%，训练速度提升3.7倍。我们的方法为基于LLM的高效表示学习建立了一种可扩展的架构无关范式。