In recent years, large language models (LLMs) have made significant progress in natural language processing (NLP), with models like ChatGPT and GPT-4 achieving impressive capabilities in various linguistic tasks. However, training models on such a large scale is challenging, and finding datasets that match the model's scale is often difficult. Fine-tuning and training models with fewer parameters using novel methods have emerged as promising approaches to overcome these challenges. One such model is MiniGPT-4, which achieves comparable vision-language understanding to GPT-4 by leveraging novel pre-training models and innovative training strategies. However, the model still faces some challenges in image understanding, particularly in artistic pictures. A novel multimodal model called ArtGPT-4 has been proposed to address these limitations. ArtGPT-4 was trained on image-text pairs using a Tesla A100 device in just 2 hours, using only about 200 GB of data. The model can depict images with an artistic flair and generate visual code, including aesthetically pleasing HTML/CSS web pages. Furthermore, the article proposes novel benchmarks for evaluating the performance of vision-language models. In the subsequent evaluation methods, ArtGPT-4 scored more than 1 point higher than the current \textbf{state-of-the-art} model and was only 0.25 points lower than artists on a 6-point scale. Our code and pre-trained model are available at \url{https://huggingface.co/Tyrannosaurus/ArtGPT-4}.
翻译:近年来,大型语言模型在自然语言处理领域取得了显著进展,ChatGPT和GPT-4等模型在各种语言任务中展现出令人印象深刻的能力。然而,在如此大规模上训练模型颇具挑战性,且通常难以找到与模型规模相匹配的数据集。利用新颖方法对参数更少的模型进行微调和训练已成为克服这些挑战的有效途径。MiniGPT-4便是此类模型之一,它通过利用新型预训练模型和创新训练策略,实现了与GPT-4相当的视觉-语言理解能力。然而,该模型在图像理解方面仍面临一些挑战,尤其是在艺术图片方面。为克服这些局限,本文提出了一种名为ArtGPT-4的新型多模态模型。ArtGPT-4在Tesla A100设备上仅用约200 GB数据,在2小时内完成图像-文本对训练。该模型能以艺术风格描绘图像,并生成视觉代码,包括美观的HTML/CSS网页。此外,本文提出了用于评估视觉-语言模型性能的新基准。在后续评估方法中,ArtGPT-4在6分量表上的得分比当前最先进模型高1分以上,仅比艺术家低0.25分。我们的代码和预训练模型可在\url{https://huggingface.co/Tyrannosaurus/ArtGPT-4}获取。