The meaning conveyed by a sentence often depends on the context in which it appears. Despite the progress of sentence embedding methods, it remains unclear how to best modify a sentence embedding conditioned on its context. To address this problem, we propose Condition-Aware Sentence Embeddings (CASE), an efficient and accurate method to create an embedding for a sentence under a given condition. First, CASE creates an embedding for the condition using a Large Language Model (LLM), where the sentence influences the attention scores computed for the tokens in the condition during pooling. Next, a supervised nonlinear projection is learned to reduce the dimensionality of the LLM-based text embeddings. We show that CASE significantly outperforms previously proposed Conditional Semantic Textual Similarity (C-STS) methods on an existing standard benchmark dataset. We find that subtracting the condition embedding consistently improves the C-STS performance of LLM-based text embeddings. Moreover, we propose a supervised dimensionality reduction method that not only reduces the dimensionality of LLM-based embeddings but also significantly improves their performance.

翻译：句子所传达的意义通常取决于其出现的上下文。尽管句子嵌入方法已取得进展，但如何根据上下文最优地调整句子嵌入仍不明确。为解决此问题，我们提出条件感知句子嵌入（CASE），这是一种在给定条件下为句子创建嵌入的高效且准确的方法。首先，CASE利用大语言模型（LLM）为条件创建嵌入，其中句子会影响池化过程中为条件标记计算的注意力分数。随后，通过监督非线性投影学习来降低基于LLM的文本嵌入的维度。实验表明，在现有标准基准数据集上，CASE显著优于先前提出的条件语义文本相似度（C-STS）方法。我们发现，减去条件嵌入能持续提升基于LLM的文本嵌入在C-STS任务上的性能。此外，我们提出的监督降维方法不仅能降低基于LLM的嵌入维度，还能显著提升其性能。