In this report, we introduce pplx-embed, a family of multilingual embedding models that employ multi-stage contrastive learning on a diffusion-pretrained language model backbone for web-scale retrieval. By leveraging bidirectional attention through diffusion-based pretraining, our models capture comprehensive bidirectional context within passages, enabling the use of mean pooling and a late chunking strategy to better preserve global context across long documents. We release two model types: pplx-embed-v1 for standard retrieval, and pplx-embed-context-v1 for contextualized embeddings that incorporate global document context into passage representations. pplx-embed-v1 achieves competitive performance on the MTEB(Multilingual, v2), MTEB(Code), MIRACL, BERGEN, and ToolRet retrieval benchmarks, while pplx-embed-context-v1 sets new records on the ConTEB benchmark. Beyond public benchmarks, pplx-embed-v1 demonstrates strong performance on our internal evaluation suite, which focuses on real-world, large-scale search scenarios over tens of millions of documents. These results validate the models' effectiveness in production environments where retrieval quality and efficiency are critical at scale.

翻译：本报告介绍了pplx-embed系列多语言嵌入模型，该系列模型采用基于扩散预训练的语言模型主干，通过多阶段对比学习实现网络规模检索。通过扩散预训练的双向注意力机制，我们的模型能够捕捉文本片段内全面的双向上下文信息，从而支持使用均值池化和延迟分块策略，以更好地在长文档中保持全局上下文。我们发布了两类模型：用于标准检索的pplx-embed-v1，以及用于上下文嵌入的pplx-embed-context-v1——后者将全局文档上下文整合到段落表征中。pplx-embed-v1在MTEB（多语言，v2）、MTEB（代码）、MIRACL、BERGEN和ToolRet检索基准测试中表现出竞争力，而pplx-embed-context-v1则在ConTEB基准测试中创造了新记录。在公开基准测试之外，pplx-embed-v1在我们内部评估体系中也展现出强劲性能，该评估重点关注数千万文档规模的真实世界大规模搜索场景。这些结果验证了模型在生产环境中的有效性，其中检索质量与大规模处理效率至关重要。