We would like to explore how morphemes can affect the performance of a language model. We trained GPT-2 and Bert model with StateMorph for both Finnish and Russian, which is a morpheme segmenting algorithm. As a comparison, we also trained a model with BPE and Morfessor. Our preliminary result shows that StateMorph can help the model to converge more efficiently and achieve a better validation score.

翻译：我们旨在探究语素对语言模型性能的影响。我们使用语素分割算法StateMorph训练了芬兰语和俄语的GPT-2和BERT模型。作为对比，我们还分别采用BPE和Morfessor训练了相应模型。初步结果表明，StateMorph能帮助模型更高效地收敛，并取得更优的验证分数。