In the past few years we have seen the meteoric appearance of dozens of foundation models of the Transformer family, all of which have memorable and sometimes funny, but not self-explanatory, names. The goal of this paper is to offer a somewhat comprehensive but simple catalog and classification of the most popular Transformer models. The paper also includes an introduction to the most important aspects and innovations in Transformer models. Our catalog will include models that are trained using self-supervised learning (e.g., BERT or GPT3) as well as those that are further trained using a human-in-the-loop (e.g. the InstructGPT model used by ChatGPT).

翻译：过去几年间，Transformer家族涌现出数十种基础模型，它们拥有令人难忘甚至趣味盎然却并非不言自明的名称。本文旨在提供一份较为全面且简洁的分类目录，涵盖最流行的Transformer模型。文章同时介绍了Transformer模型中最关键的方面与创新。我们的目录既包含基于自监督学习训练的模型（例如BERT或GPT3），也包括通过人机交互循环进一步训练的模型（如ChatGPT所采用的InstructGPT模型）。