We would like to explore how morphemes can affect the performance of a language model. We trained GPT-2 and Bert model with StateMorph for both Finnish and Russian, which is a morpheme segmenting algorithm. As a comparison, we also trained a model with BPE and Morfessor. Our preliminary result shows that StateMorph can help the model to converge more efficiently and achieve a better validation score.

翻译：我们旨在探索语素如何影响语言模型的性能。我们使用StateMorph（一种语素切分算法）训练了针对芬兰语和俄语的GPT-2和BERT模型。作为对比，我们还使用BPE和Morfessor训练了模型。初步结果表明，StateMorph能够帮助模型更高效地收敛，并取得更好的验证分数。