Prominent works in the field of Natural Language Processing have long attempted to create new innovative models by improving upon previous model training approaches, altering model architecture, and developing more in-depth datasets to better their performance. However, with the quickly advancing field of NLP comes increased greenhouse gas emissions, posing concerns over the environmental damage caused by training LLMs. Gaining a comprehensive understanding of the various costs, particularly those pertaining to environmental aspects, that are associated with artificial intelligence serves as the foundational basis for ensuring safe AI models. Currently, investigations into the CO2 emissions of AI models remain an emerging area of research, and as such, in this paper, we evaluate the CO2 emissions of well-known large language models, which have an especially high carbon footprint due to their significant amount of model parameters. We argue for the training of LLMs in a way that is responsible and sustainable by suggesting measures for reducing carbon emissions. Furthermore, we discuss how the choice of hardware affects CO2 emissions by contrasting the CO2 emissions during model training for two widely used GPUs. Based on our results, we present the benefits and drawbacks of our proposed solutions and make the argument for the possibility of training more environmentally safe AI models without sacrificing their robustness and performance.

翻译：自然语言处理领域的杰出工作长期以来一直致力于通过改进先前模型训练方法、改变模型架构以及开发更深入的数据集来创造新的创新模型，从而提升其性能。然而，随着NLP领域的快速进步，温室气体排放也随之增加，引发了人们对训练大语言模型所造成环境损害的担忧。全面了解与人工智能相关的各种成本，特别是那些涉及环境方面的成本，是为确保AI模型安全奠定基础。目前，对AI模型二氧化碳排放的研究仍是一个新兴的研究领域，因此，在本文中，我们评估了知名大语言模型的二氧化碳排放，这些模型因包含大量参数而具有特别高的碳足迹。我们主张通过提出减少碳排放的措施，以负责任和可持续的方式训练大语言模型。此外，我们通过对比两种广泛使用的图形处理器在模型训练期间的二氧化碳排放，讨论了硬件选择对碳排放的影响。基于我们的研究结果，我们展示了所提出解决方案的优缺点，并论证了在不牺牲其鲁棒性和性能的情况下训练更环保AI模型的可能性。