The multilingual Sentence-BERT (SBERT) models map different languages to common representation space and are useful for cross-language similarity and mining tasks. We propose a simple yet effective approach to convert vanilla multilingual BERT models into multilingual sentence BERT models using synthetic corpus. We simply aggregate translated NLI or STS datasets of the low-resource target languages together and perform SBERT-like fine-tuning of the vanilla multilingual BERT model. We show that multilingual BERT models are inherent cross-lingual learners and this simple baseline fine-tuning approach without explicit cross-lingual training yields exceptional cross-lingual properties. We show the efficacy of our approach on 10 major Indic languages and also show the applicability of our approach to non-Indic languages German and French. Using this approach, we further present L3Cube-IndicSBERT, the first multilingual sentence representation model specifically for Indian languages Hindi, Marathi, Kannada, Telugu, Malayalam, Tamil, Gujarati, Odia, Bengali, and Punjabi. The IndicSBERT exhibits strong cross-lingual capabilities and performs significantly better than alternatives like LaBSE, LASER, and paraphrase-multilingual-mpnet-base-v2 on Indic cross-lingual and monolingual sentence similarity tasks. We also release monolingual SBERT models for each of the languages and show that IndicSBERT performs competitively with its monolingual counterparts. These models have been evaluated using embedding similarity scores and classification accuracy.

翻译：多语言句子BERT（SBERT）模型将不同语言映射到共同表示空间，并适用于跨语言相似性计算与挖掘任务。我们提出一种简单有效的方法，利用合成语料将普通多语言BERT模型转化为多语言句子BERT模型。该方法仅需将低资源目标语言的已翻译NLI或STS数据集合并，并对普通多语言BERT模型进行SBERT风格的微调。实验表明，多语言BERT模型本身具有跨语言学习能力，这种无需显式跨语言训练的简单基线微调方法即可获得优异的跨语言特性。我们在10种主要印度语言上验证了该方法的有效性，并展示了其对非印度语言（如德语和法语）的适用性。通过该方法，我们进一步推出L3Cube-IndicSBERT——首个专为印度语种（印地语、马拉地语、卡纳达语、泰卢固语、马拉雅拉姆语、泰米尔语、古吉拉特语、奥里亚语、孟加拉语、旁遮普语）设计的的多语言句子表示模型。IndicSBERT展现出强大的跨语言能力，在印度语言跨语言及单语句子相似性任务上显著优于LaBSE、LASER及paraphrase-multilingual-mpnet-base-v2等现有方案。我们还发布了各语言的单语SBERT模型，并证明IndicSBERT与其单语版本性能相当。这些模型已通过嵌入相似度评分与分类准确率进行评估。