Deep learning-based and lately Transformer-based language models have been dominating the studies of natural language processing in the last years. Thanks to their accurate and fast fine-tuning characteristics, they have outperformed traditional machine learning-based approaches and achieved state-of-the-art results for many challenging natural language understanding (NLU) problems. Recent studies showed that the Transformer-based models such as BERT, which is Bidirectional Encoder Representations from Transformers, have reached impressive achievements on many tasks. Moreover, thanks to their transfer learning capacity, these architectures allow us to transfer pre-built models and fine-tune them to specific NLU tasks such as question answering. In this study, we provide a Transformer-based model and a baseline benchmark for the Turkish Language. We successfully fine-tuned a Turkish BERT model, namely BERTurk that is trained with base settings, to many downstream tasks and evaluated with a the Turkish Benchmark dataset. We showed that our studies significantly outperformed other existing baseline approaches for Named-Entity Recognition, Sentiment Analysis, Question Answering and Text Classification in Turkish Language. We publicly released these four fine-tuned models and resources in reproducibility and with the view of supporting other Turkish researchers and applications.

翻译：基于深度学习以及近年来基于Transformer的语言模型，在过去几年中主导了自然语言处理的研究。凭借其准确且快速的微调特性，这些模型超越了传统的基于机器学习的方法，并在许多具有挑战性的自然语言理解（NLU）问题上取得了最先进的成果。最近的研究表明，诸如BERT（来自Transformers的双向编码器表示）等基于Transformer的模型，已在许多任务上取得了令人瞩目的成就。此外，得益于其迁移学习能力，这些架构允许我们迁移预构建的模型并将其微调至特定的NLU任务，例如问答系统。在本研究中，我们为土耳其语提供了一个基于Transformer的模型和基线基准。我们成功地将一个以基础设置训练的土耳其语BERT模型——即BERTurk——微调至多个下游任务，并使用土耳其语基准数据集进行了评估。我们的研究表明，在土耳其语的命名实体识别、情感分析、问答系统和文本分类任务中，我们的研究显著优于其他现有的基线方法。为了实现可重复性并支持其他土耳其研究人员和应用，我们公开发布了这四个微调后的模型及资源。