BatGPT is a large-scale language model designed and trained jointly by Wuhan University and Shanghai Jiao Tong University. It is capable of generating highly natural and fluent text in response to various types of input, including text prompts, images, and audio. In the modeling level, we employ a bidirectional autoregressive architecture that allows the model to efficiently capture the complex dependencies of natural language, making it highly effective in tasks such as language generation, dialog systems, and question answering. Moreover, the bidirectional autoregressive modeling not only operates from left to right but also from right to left, effectively reducing fixed memory effects and alleviating model hallucinations. In the training aspect, we propose a novel parameter expansion method for leveraging the pre-training of smaller models and employ reinforcement learning from both AI and human feedback, aimed at improving the model's alignment performance. Overall, these approaches significantly improve the effectiveness of BatGPT, and the model can be utilized for a wide range of natural language applications.

翻译：BatGPT是由武汉大学和上海交通大学联合设计与训练的大规模语言模型。它能针对文本提示、图像和音频等多种输入类型，生成高度自然流畅的文本。在建模层面，我们采用双向自回归架构，使模型能够高效捕捉自然语言的复杂依赖关系，从而在语言生成、对话系统和问答等任务中表现出色。此外，双向自回归建模不仅从左到右运行，也从右到左操作，有效减少了固定记忆效应并缓解了模型幻觉问题。在训练方面，我们提出了一种新颖的参数扩展方法，用于利用较小模型的预训练成果，并采用基于人工智能和人类反馈的强化学习，旨在提升模型的对齐性能。总体而言，这些方法显著提高了BatGPT的有效性，该模型可广泛应用于各类自然语言处理任务。