In this report, we introduce pplx-embed, a family of multilingual embedding models that employ multi-stage contrastive learning on a diffusion-pretrained language model backbone for web-scale retrieval. By leveraging bidirectional attention through diffusion-based pretraining, our models capture comprehensive bidirectional context within passages, enabling the use of mean pooling and a late chunking strategy to better preserve global context across long documents. We release two model types: pplx-embed-v1 for standard retrieval, and pplx-embed-context-v1 for contextualized embeddings that incorporate global document context into passage representations. pplx-embed-v1 achieves competitive performance on the MTEB(Multilingual, v2), MTEB(Code), MIRACL, BERGEN, and ToolRet retrieval benchmarks, while pplx-embed-context-v1 sets new records on the ConTEB benchmark. Beyond public benchmarks, pplx-embed-v1 demonstrates strong performance on our internal evaluation suite, focusing on real-world, large-scale search scenarios constructed from 1B production web pages. These results validate the models' effectiveness in production environments where retrieval quality and efficiency are critical at scale.

翻译：本报告介绍了pplx-embed系列多语言嵌入模型，该系列模型采用基于扩散预训练语言模型骨干网络，通过多阶段对比学习实现网络级检索。通过扩散预训练获得的双向注意力机制，我们的模型能够捕捉文本片段内完整的双向上下文信息，从而支持均值池化与延迟分块策略，以更好地保留长文档的全局上下文。我们发布了两类模型：用于标准检索的pplx-embed-v1，以及用于上下文嵌入的pplx-embed-context-v1——后者将全局文档上下文整合到段落表征中。pplx-embed-v1在MTEB（多语言v2版）、MTEB（代码）、MIRACL、BERGEN和ToolRet检索基准测试中均取得具有竞争力的性能，而pplx-embed-context-v1则在ConTEB基准测试中创造了新纪录。除公开基准测试外，pplx-embed-v1在我们基于10亿级生产网页构建的、专注于现实世界大规模搜索场景的内部评估体系中亦表现优异。这些结果验证了模型在生产环境中的有效性，其中检索质量与大规模处理效率至关重要。