We open-source a state-of-the-art 4B-parameter generative model series for Vietnamese, which includes the base pre-trained monolingual model PhoGPT-4B and its chat variant, PhoGPT-4B-Chat. The base model, PhoGPT-4B, with exactly 3.7B parameters, is pre-trained from scratch on a Vietnamese corpus of 102B tokens, with an 8192 context length, employing a vocabulary of 20480 token types. The chat variant, PhoGPT-4B-Chat, is the modeling output obtained by fine-tuning PhoGPT-4B on a dataset of 70K instructional prompts and their responses, along with an additional 290K conversations. We demonstrate its strong performance compared to previous closed-source and open-source 7B-parameter models. Our PhoGPT models are available at: https://github.com/VinAIResearch/PhoGPT

翻译：我们开源了一个面向越南语的最先进4B参数生成模型系列，包括基础预训练单语模型PhoGPT-4B及其对话变体PhoGPT-4B-Chat。基础模型PhoGPT-4B精确包含37亿参数，基于包含1020亿词元的越南语语料库从头预训练，支持8192词元的上下文长度，采用20480种词元类型的词汇表。对话变体PhoGPT-4B-Chat通过在7万条指令提示及其响应数据（另加29万段对话）上微调PhoGPT-4B获得的建模输出。我们展示了其相比之前闭源和开源7B参数模型的强劲性能。我们的PhoGPT模型可通过以下链接获取：https://github.com/VinAIResearch/PhoGPT