We present the first experiments on Native Language Identification (NLI) using LLMs such as GPT-4. NLI is the task of predicting a writer's first language by analyzing their writings in a second language, and is used in second language acquisition and forensic linguistics. Our results show that GPT models are proficient at NLI classification, with GPT-4 setting a new performance record of 91.7% on the benchmark TOEFL11 test set in a zero-shot setting. We also show that unlike previous fully-supervised settings, LLMs can perform NLI without being limited to a set of known classes, which has practical implications for real-world applications. Finally, we also show that LLMs can provide justification for their choices, providing reasoning based on spelling errors, syntactic patterns, and usage of directly translated linguistic patterns.

翻译：我们首次提出了使用GPT-4等大语言模型（LLMs）进行母语识别（NLI）的实验。母语识别任务旨在通过分析作者在第二语言中的写作内容来预测其母语，广泛应用于第二语言习得和司法语言学领域。结果表明，GPT模型在NLI分类任务中表现优异：在零样本设置下，GPT-4在基准测试集TOEFL11上创下了91.7%的新性能纪录。此外，与先前全监督设置不同，LLMs能够在不限于已知类别集合的情况下执行NLI，这在现实应用中具有实际意义。最后，我们还发现LLMs能够为其选择提供解释，其推理依据包括拼写错误、句法模式以及直接翻译的语言模式的使用。