Text embedding models are widely used for semantic similarity tasks, including information retrieval, clustering, and classification. General-purpose models are typically trained with single- or multi-stage processes using contrastive loss functions. We introduce a novel training regimen that combines model distillation techniques with task-specific contrastive loss to produce compact, high-performance embedding models. Our findings suggest that this approach is more effective for training small models than purely contrastive or distillation-based training paradigms alone. Benchmark scores for the resulting models, jina-embeddings-v5-text-small and jina-embeddings-v5-text-nano, exceed or match the state-of-the-art for models of similar size. jina-embeddings-v5-text models additionally support long texts (up to 32k tokens) in many languages, and generate embeddings that remain robust under truncation and binary quantization. Model weights are publicly available, hopefully inspiring further advances in embedding model development.

翻译：文本嵌入模型广泛应用于语义相似性任务，包括信息检索、聚类和分类。通用模型通常采用单阶段或多阶段流程，并使用对比损失函数进行训练。我们引入了一种新的训练方案，该方案结合了模型蒸馏技术与任务特定对比损失，以生成紧凑且高性能的嵌入模型。我们的研究表明，相较于单独使用纯对比训练或纯蒸馏训练范式，该方法在训练小型模型时更为有效。所得模型jina-embeddings-v5-text-small和jina-embeddings-v5-text-nano的基准测试得分达到或超越了同等规模模型的最优水平。jina-embeddings-v5-text模型还支持多语言长文本（最长32k词元），并在截断和二元量化条件下仍能生成稳健的嵌入。模型权重已公开，有望进一步推动嵌入模型开发领域的进展。