Sentence embeddings enable us to capture the semantic similarity of short texts. Most sentence embedding models are trained for general semantic textual similarity tasks. Therefore, to use sentence embeddings in a particular domain, the model must be adapted to it in order to achieve good results. Usually, this is done by fine-tuning the entire sentence embedding model for the domain of interest. While this approach yields state-of-the-art results, all of the model's weights are updated during fine-tuning, making this method resource-intensive. Therefore, instead of fine-tuning entire sentence embedding models for each target domain individually, we propose to train lightweight adapters. These domain-specific adapters do not require fine-tuning all underlying sentence embedding model parameters. Instead, we only train a small number of additional parameters while keeping the weights of the underlying sentence embedding model fixed. Training domain-specific adapters allows always using the same base model and only exchanging the domain-specific adapters to adapt sentence embeddings to a specific domain. We show that using adapters for parameter-efficient domain adaptation of sentence embeddings yields competitive performance within 1% of a domain-adapted, entirely fine-tuned sentence embedding model while only training approximately 3.6% of the parameters.

翻译：句嵌入使我们能够捕捉短文本的语义相似性。大多数句嵌入模型是为通用语义文本相似性任务训练的。因此，为了在特定领域中使用句嵌入，必须对该模型进行适配以获得良好效果。通常，这通过微调整个句嵌入模型以适应目标领域来实现。虽然这种方法能取得最先进的结果，但微调过程中会更新模型的所有权重，使得该方法资源密集。因此，我们提出训练轻量级适配器，而非为每个目标领域单独微调整个句嵌入模型。这些领域特定的适配器无需微调底层句嵌入模型的所有参数。相反，我们仅训练少量额外参数，同时保持底层句嵌入模型的权重固定。训练领域特定的适配器使得可以始终使用相同的基模型，仅通过交换领域特定的适配器来将句嵌入适配到特定领域。我们证明，使用适配器进行句嵌入的参数高效领域适配，在性能上与完全微调的领域适配句嵌入模型相差1%以内，同时仅训练约3.6%的参数。