We open-source a state-of-the-art 4B-parameter generative model series for Vietnamese, which includes the base pre-trained monolingual model PhoGPT-4B and its chat variant, PhoGPT-4B-Chat. The base model, PhoGPT-4B, with exactly 3.7B parameters, is pre-trained from scratch on a Vietnamese corpus of 102B tokens, with an 8192 context length, employing a vocabulary of 20480 token types. The chat variant, PhoGPT-4B-Chat, is the modeling output obtained by fine-tuning PhoGPT-4B on a dataset of 70K instructional prompts and their responses, along with an additional 290K conversations. In addition, we also demonstrate its superior performance compared to previous open-source models. Our PhoGPT models are available at: https://github.com/VinAIResearch/PhoGPT

翻译：我们开源了一系列针对越南语的先进40亿参数生成式模型，包括基础预训练单语模型PhoGPT-4B及其对话变体PhoGPT-4B-Chat。基础模型PhoGPT-4B（实际参数量为37亿）使用包含1020亿词元的越南语语料库从头开始预训练，支持8192上下文长度，采用20480种词元类型组成的词表。对话变体PhoGPT-4B-Chat是通过在7万条指令提示及其响应数据集以及额外29万条对话数据集上微调PhoGPT-4B获得的建模输出。此外，我们还展示了该模型相较于此前开源模型的优越性能。我们的PhoGPT模型可通过以下链接获取：https://github.com/VinAIResearch/PhoGPT