Learning multi-lingual sentence embeddings is a fundamental task in natural language processing. Recent trends in learning both mono-lingual and multi-lingual sentence embeddings are mainly based on contrastive learning (CL) among an anchor, one positive, and multiple negative instances. In this work, we argue that leveraging multiple positives should be considered for multi-lingual sentence embeddings because (1) positives in a diverse set of languages can benefit cross-lingual learning, and (2) transitive similarity across multiple positives can provide reliable structural information for learning. In order to investigate the impact of multiple positives in CL, we propose a novel approach, named MPCL, to effectively utilize multiple positive instances to improve the learning of multi-lingual sentence embeddings. Experimental results on various backbone models and downstream tasks demonstrate that MPCL leads to better retrieval, semantic similarity, and classification performances compared to conventional CL. We also observe that in unseen languages, sentence embedding models trained on multiple positives show better cross-lingual transfer performance than models trained on a single positive instance.

翻译：多语言句子嵌入是自然语言处理中的一项基础任务。当前单语与多语言句子嵌入的学习方法主要基于对比学习，其核心结构包含锚点、一个正例和多个负例。本研究认为，在多语言句子嵌入中应充分利用多个正例，原因在于：(1)来自不同语言的正例有助于跨语言学习；(2)多个正例间的传递相似性可为学习过程提供可靠的结构信息。为探究多正例在对比学习中的影响，我们提出名为MPCL的新型方法，通过有效利用多个正例来优化多语言句子嵌入的学习。在多种骨干模型与下游任务上的实验结果表明，与传统对比学习相比，MPCL在检索、语义相似度及分类任务中均取得更优性能。此外，我们观察到在未见语言上，基于多正例训练的句子嵌入模型比基于单正例训练的模型展现出更强的跨语言迁移能力。