Transformer based pre-trained models such as BERT and its variants, which are trained on large corpora, have demonstrated tremendous success for natural language processing (NLP) tasks. Most of academic works are based on the English language; however, the number of multilingual and language specific studies increase steadily. Furthermore, several studies claimed that language specific models outperform multilingual models in various tasks. Therefore, the community tends to train or fine-tune the models for the language of their case study, specifically. In this paper, we focus on Turkish maps data and thoroughly evaluate both multilingual and Turkish based BERT, DistilBERT, ELECTRA and RoBERTa. Besides, we also propose a MultiLayer Perceptron (MLP) for fine-tuning BERT in addition to the standard approach of one-layer fine-tuning. For the dataset, a mid-sized Address Parsing corpus taken with a relatively high quality is constructed. Conducted experiments on this dataset indicate that Turkish language specific models with MLP fine-tuning yields slightly better results when compared to the multilingual fine-tuned models. Moreover, visualization of address tokens' representations further indicates the effectiveness of BERT variants for classifying a variety of addresses.

翻译：基于Transformer的预训练模型（如BERT及其变体）在大规模语料上训练后，在自然语言处理（NLP）任务中展现出显著成功。大多数研究工作以英语为基础，然而多语言及特定语言研究的数量正稳步增长。此外，多项研究声称特定语言模型在多任务上优于多语言模型。因此，研究社区倾向于针对其案例研究的语言专门训练或微调模型。本文聚焦土耳其地图数据，全面评估了多语言及基于土耳其语的BERT、DistilBERT、ELECTRA和RoBERTa模型。同时，除单层微调的标准方法外，我们提出了一种用于微调BERT的多层感知器（MLP）。数据集方面，我们构建了一个质量相对较高的中等规模地址解析语料库。在该数据集上的实验表明，采用MLP微调的土耳其语特定模型相比多语言微调模型取得了略优的结果。此外，地址令牌表征的可视化进一步展示了BERT变体在各类地址分类中的有效性。