Large Language Models (LLMs) exhibit impressive reasoning and data augmentation capabilities in various NLP tasks. However, what about small models? In this work, we propose TeacherLM-7.1B, capable of annotating relevant fundamentals, chain of thought, and common mistakes for most NLP samples, which makes annotation more than just an answer, thus allowing other models to learn "why" instead of just "what". The TeacherLM-7.1B model achieved a zero-shot score of 52.3 on MMLU, surpassing most models with over 100B parameters. Even more remarkable is its data augmentation ability. Based on TeacherLM-7.1B, we augmented 58 NLP datasets and taught various student models with different parameters from OPT and BLOOM series in a multi-task setting. The experimental results indicate that the data augmentation provided by TeacherLM has brought significant benefits. We will release the TeacherLM series of models and augmented datasets as open-source.

翻译：大型语言模型（LLMs）在各种自然语言处理任务中展现出令人印象深刻的推理和数据增强能力。然而，小模型的表现如何？本文提出TeacherLM-7.1B模型，能够为大多数自然语言处理样本标注相关基础知识、思维链和常见错误，使标注不再仅仅提供答案，从而让其他模型学会“为什么”而非仅仅“是什么”。该模型在MMLU上实现了52.3的零样本得分，超越了大量参数超过100B的模型。更令人瞩目的是其数据增强能力。基于TeacherLM-7.1B，我们扩充了58个自然语言处理数据集，并在多任务设置下，使用OPT和BLOOM系列中不同参数的多种学生模型进行训练。实验结果表明，TeacherLM提供的数据增强带来了显著收益。我们将开源TeacherLM系列模型及扩充后的数据集。