Contextual language models have been trained on Classical languages, including Ancient Greek and Latin, for tasks such as lemmatization, morphological tagging, part of speech tagging, authorship attribution, and detection of scribal errors. However, high-quality sentence embedding models for these historical languages are significantly more difficult to achieve due to the lack of training data. In this work, we use a multilingual knowledge distillation approach to train BERT models to produce sentence embeddings for Ancient Greek text. The state-of-the-art sentence embedding approaches for high-resource languages use massive datasets, but our distillation approach allows our Ancient Greek models to inherit the properties of these models while using a relatively small amount of translated sentence data. We build a parallel sentence dataset using a sentence-embedding alignment method to align Ancient Greek documents with English translations, and use this dataset to train our models. We evaluate our models on translation search, semantic similarity, and semantic retrieval tasks and investigate translation bias. We make our training and evaluation datasets freely available at https://github.com/kevinkrahn/ancient-greek-datasets .

翻译：上下文语言模型已针对古典语言（包括古希腊语和拉丁语）进行训练，用于词形还原、词性标注、作者归属以及手抄错误检测等任务。然而，由于训练数据的匮乏，为这些历史语言构建高质量的句子嵌入模型难度显著增大。在本研究中，我们采用多语言知识蒸馏方法，训练BERT模型以生成古希腊语文本的句子嵌入。面向高资源语言的最先进句子嵌入方法依赖大规模数据集，而我们的蒸馏方法允许古希腊语模型在仅使用相对少量翻译句子数据的情况下，继承这些模型的特性。我们利用一种句子嵌入对齐方法，构建了将古希腊语文档与英文翻译对齐的平行句子数据集，并基于该数据集训练模型。我们在翻译搜索、语义相似度及语义检索任务上评估模型性能，并探究翻译偏差的影响。我们已将训练与评估数据集免费开放至 https://github.com/kevinkrahn/ancient-greek-datasets 。