To be discoverable in an embedding-based search process, each part of a document should be reflected in its embedding representation. To quantify any potential reflection biases, we introduce a permutation-based evaluation framework. With this, we observe that state-of-the-art embedding models exhibit systematic positional and language biases when documents are longer and consist of multiple segments. Specifically, early segments and segments in higher-resource languages like English are over-represented, while later segments and segments in lower-resource languages are marginalized. In our further analysis, we find that the positional bias stems from front-loaded attention distributions in pooling-token embeddings, where early tokens receive more attention. To mitigate this issue, we introduce an inference-time attention calibration method that redistributes attention more evenly across document positions, increasing discoverabiltiy of later segments. Our evaluation framework and attention calibration is available at https://github.com/impresso/fair-sentence-transformers

翻译：为了使文档的每一部分在基于嵌入的搜索过程中都能被发现，其嵌入表示应充分反映文档的各个部分。为量化任何潜在的反映偏差，我们引入了一种基于排列的评估框架。通过该框架，我们观察到当文档较长且由多个片段组成时，最先进的嵌入模型会表现出系统性的位置偏差和语言偏差。具体而言，早期片段以及英语等高资源语言片段被过度表征，而后期片段和低资源语言片段则被边缘化。在进一步分析中，我们发现位置偏差源于池化标记嵌入中前向集中的注意力分布，即早期标记获得了更多关注。为缓解此问题，我们提出了一种推理时注意力校准方法，该方法能在文档的不同位置更均匀地重新分配注意力，从而提高后期片段的可发现性。我们的评估框架和注意力校准方法可在 https://github.com/impresso/fair-sentence-transformers 获取。