Chatbots based on Large Language Models (LLMs) have shown strong capabilities in language understanding. In this study, we explore the potential of LLMs in assisting corpus-based linguistic studies through automatic annotation of texts with specific categories of linguistic information. Specifically, we examined to what extent LLMs understand the functional elements constituting the speech act of apology from a local grammar perspective, by comparing the performance of ChatGPT (powered by GPT-3.5), Bing chatbot (powered by GPT-4), and a human coder in the annotation task. The results demonstrate that Bing chatbot significantly outperformed ChatGPT in the task. Compared to human annotator, the overall performance of Bing chatbot was slightly less satisfactory. However, it already achieved high F1 scores: 99.95% for the tag of APOLOGISING, 91.91% for REASON, 95.35% for APOLOGISER, 89.74% for APOLOGISEE, and 96.47% for INTENSIFIER. Therefore, we propose that LLM-assisted annotation is a promising automated approach for corpus studies.

翻译：基于大语言模型（LLMs）的聊天机器人在语言理解方面展现出强大能力。本研究通过利用大语言模型自动标注特定语言信息类别，探索其在辅助基于语料库的语言学研究中的潜力。具体而言，我们从局部语法视角出发，对比ChatGPT（基于GPT-3.5）、Bing聊天机器人（基于GPT-4）以及人类编码者在标注任务中的表现，考察大语言模型在多大程度上理解构成道歉言语行为的功能要素。结果表明，Bing聊天机器人在该任务中显著优于ChatGPT。与人类标注者相比，Bing聊天机器人的整体表现略逊一筹，但已取得较高F1分数：APOLOGISING标签达99.95%，REASON标签达91.91%，APOLOGISER标签达95.35%，APOLOGISEE标签达89.74%，INTENSIFIER标签达96.47%。因此，我们认为大语言模型辅助标注是一种具有前景的语料库研究自动化方法。