The ability to extract high-quality translation dictionaries from monolingual word embedding spaces depends critically on the geometric similarity of the spaces -- their degree of "isomorphism." We address the root-cause of faulty cross-lingual mapping: that word embedding training resulted in the underlying spaces being non-isomorphic. We incorporate global measures of isomorphism directly into the Skip-gram loss function, successfully increasing the relative isomorphism of trained word embedding spaces and improving their ability to be mapped to a shared cross-lingual space. The result is improved bilingual lexicon induction in general data conditions, under domain mismatch, and with training algorithm dissimilarities. We release IsoVec at https://github.com/kellymarchisio/isovec.
翻译:从单语词嵌入空间中提取高质量翻译词典的能力,关键取决于这些空间的几何相似性——即它们的"同构"程度。我们针对跨语言映射失效的根本原因展开研究:词嵌入训练导致底层空间非对偶同构。通过将全局同构度量直接纳入Skip-gram损失函数,我们成功提升了训练所得词嵌入空间的相对同构性,增强了它们映射至共享跨语言空间的能力。该方法在一般数据条件、领域不匹配及训练算法差异场景下,均能有效改善双语词典归纳性能。IsoVec工具已开源发布于https://github.com/kellymarchisio/isovec。