Sentence embeddings enable us to capture the semantic similarity of short texts. Most sentence embedding models are trained for general semantic textual similarity (STS) tasks. Therefore, to use sentence embeddings in a particular domain, the model must be adapted to it in order to achieve good results. Usually, this is done by fine-tuning the entire sentence embedding model for the domain of interest. While this approach yields state-of-the-art results, all of the model's weights are updated during fine-tuning, making this method resource-intensive. Therefore, instead of fine-tuning entire sentence embedding models for each target domain individually, we propose to train lightweight adapters. These domain-specific adapters do not require fine-tuning all underlying sentence embedding model parameters. Instead, we only train a small number of additional parameters while keeping the weights of the underlying sentence embedding model fixed. Training domain-specific adapters allows always using the same base model and only exchanging the domain-specific adapters to adapt sentence embeddings to a specific domain. We show that using adapters for parameter-efficient domain adaptation of sentence embeddings yields competitive performance within 1% of a domain-adapted, entirely fine-tuned sentence embedding model while only training approximately 3.6% of the parameters.

翻译：句子嵌入使我们能够捕捉短文本的语义相似性。大多数句子嵌入模型是针对通用语义文本相似性（STS）任务进行训练的。因此，要在特定领域中使用句子嵌入，必须对模型进行领域自适应以获得良好的效果。通常，这是通过对整个句子嵌入模型在目标领域上进行微调来实现的。尽管这种方法能取得最先进的结果，但微调过程中会更新模型的所有权重，导致该方法资源开销巨大。因此，我们提出训练轻量级适配器，而不是针对每个目标领域单独微调整个句子嵌入模型。这些领域特定的适配器无需微调底层句子嵌入模型的所有参数，相反，我们只训练少量额外参数，同时保持底层句子嵌入模型的权重固定。训练领域特定适配器使得我们可以始终使用相同的基座模型，仅需更换领域特定适配器即可将句子嵌入适应到特定领域。我们证明，使用适配器对句子嵌入进行参数高效领域自适应，其性能与完全微调的领域自适应句子嵌入模型相比仅相差1%以内，而训练参数仅为后者的约3.6%。