Large Language Models (LLMs) exhibit impressive reasoning and data augmentation capabilities in various NLP tasks. However, what about small models? In this work, we propose TeacherLM-7.1B, capable of annotating relevant fundamentals, chain of thought, and common mistakes for most NLP samples, which makes annotation more than just an answer, thus allowing other models to learn "why" instead of just "what". The TeacherLM-7.1B model achieved a zero-shot score of 52.3 on MMLU, surpassing most models with over 100B parameters. Even more remarkable is its data augmentation ability. Based on TeacherLM-7.1B, we augmented 58 NLP datasets and taught various student models with different parameters from OPT and BLOOM series in a multi-task setting. The experimental results indicate that the data augmentation provided by TeacherLM has brought significant benefits. We will release the TeacherLM series of models and augmented datasets as open-source.

翻译：大型语言模型（LLMs）在各种自然语言处理任务中展现出卓越的推理能力和数据增强能力。然而，小型模型的表现又如何呢？为此，我们提出TeacherLM-7.1B模型，它能够为大多数自然语言处理样本注释相关基础知识、思维链和常见错误，从而使标注不再局限于答案本身，让其他模型能够学习“为什么”而不仅仅是“是什么”。TeacherLM-7.1B模型在MMLU基准测试中取得了52.3分的零样本成绩，超过了多数参数量超过1000亿的模型。更令人瞩目的是其数据增强能力。基于TeacherLM-7.1B，我们增强了58个自然语言处理数据集，并在多任务设置下对来自OPT和BLOOM系列的不同参数量的学生模型进行训练。实验结果表明，TeacherLM提供的数据增强带来了显著的收益。我们将开源TeacherLM系列模型及其增强后的数据集。