BatGPT is a large-scale language model designed and trained jointly by Wuhan University and Shanghai Jiao Tong University. It is capable of generating highly natural and fluent text in response to various types of input, including text prompts, images, and audio. In the modeling level, we employ a bidirectional autoregressive architecture that allows the model to efficiently capture the complex dependencies of natural language, making it highly effective in tasks such as language generation, dialog systems, and question answering. Moreover, the bidirectional autoregressive modeling not only operates from left to right but also from right to left, effectively reducing fixed memory effects and alleviating model hallucinations. In the training aspect, we propose a novel parameter expansion method for leveraging the pre-training of smaller models and employ reinforcement learning from both AI and human feedback, aimed at improving the model's alignment performance. Overall, these approaches significantly improve the effectiveness of BatGPT, and the model can be utilized for a wide range of natural language applications.

翻译：BatGPT是由武汉大学与上海交通大学联合设计与训练的大规模语言模型。该模型能够针对文本提示、图像及音频等多种输入类型生成高度自然流畅的文本。在建模层面，我们采用双向自回归架构，使模型能够高效捕捉自然语言的复杂依赖性，在语言生成、对话系统和问答等任务中表现出色。此外，双向自回归建模不仅从左至右运行，还从右至左运行，有效降低了固定记忆效应并缓解了模型幻觉。在训练方面，我们提出了一种新颖的参数扩展方法，以利用较小模型的预训练成果，并采用来自人工智能与人类反馈的强化学习，旨在提升模型的对齐性能。总体而言，这些方法显著提升了BatGPT的有效性，该模型可广泛应用于各类自然语言处理任务。