The rising popularity of large language models (LLMs) has raised concerns about machine-generated text (MGT), particularly in academic settings, where issues like plagiarism and misinformation are prevalent. As a result, developing a highly generalizable and adaptable MGT detection system has become an urgent priority. Given that LLMs are most commonly misused in academic writing, this work investigates the generalization and adaptation capabilities of MGT detectors in three key aspects specific to academic writing: First, we construct MGT-Acedemic, a large-scale dataset comprising over 336M tokens and 749K samples. MGT-Acedemic focuses on academic writing, featuring human-written texts (HWTs) and MGTs across STEM, Humanities, and Social Sciences, paired with an extensible code framework for efficient benchmarking. Second, we benchmark the performance of various detectors for binary classification and attribution tasks in both in-domain and cross-domain settings. This benchmark reveals the often-overlooked challenges of attribution tasks. Third, we introduce a novel attribution task where models have to adapt to new classes over time without (or with very limited) access to prior training data in both few-shot and many-shot scenarios. We implement eight different adapting techniques to improve the performance and highlight the inherent complexity of the task. Our findings provide insights into the generalization and adaptation ability of MGT detectors across diverse scenarios and lay the foundation for building robust, adaptive detection systems. The code framework is available at https://github.com/Y-L-LIU/MGTBench-2.0.

翻译：大型语言模型（LLM）的日益普及引发了人们对机器生成文本（MGT）的担忧，尤其是在学术环境中，剽窃和错误信息等问题普遍存在。因此，开发一个具有高度泛化性和适应性的MGT检测系统已成为当务之急。鉴于LLM最常被滥用于学术写作，本研究从学术写作特有的三个关键方面探讨了MGT检测器的泛化与适应能力：首先，我们构建了MGT-Acedemic，这是一个包含超过3.36亿个标记和74.9万个样本的大规模数据集。MGT-Acedemic专注于学术写作，涵盖STEM、人文和社会科学领域的人类撰写文本（HWT）与MGT，并配有一个可扩展的代码框架以实现高效基准测试。其次，我们在域内和跨域设置下，对各种检测器在二分类和归属任务上的性能进行了基准测试。该基准测试揭示了归属任务中常被忽视的挑战。第三，我们引入了一种新颖的归属任务，要求模型在少样本和多样本场景下，随时间推移适应新类别，且无法（或仅能极有限地）访问先前的训练数据。我们实施了八种不同的适应技术以提升性能，并凸显了该任务固有的复杂性。我们的研究结果揭示了MGT检测器在不同场景下的泛化与适应能力，为构建鲁棒、自适应的检测系统奠定了基础。代码框架可在 https://github.com/Y-L-LIU/MGTBench-2.0 获取。