This paper analyzes the performance of Small Language Models (SLMs) and Vision Language Models (VLMs) and evaluates the trade-off between model performance and carbon emissions across 4 essential tasks: Image Captioning, Visual Question Answering (VQA), Dialogue Summarization and Text-to-SQL conversion. Various SLMs and VLMs belonging to the Qwen and LLaMA architecture family are chosen and variants based on model size in terms of the number of parameters, quantization level and fine-tuning parameters are evaluated. The model variant's performance and carbon emissions are calculated. To quantify the trade-off between model performance and carbon emissions, we introduce a novel metric called CEGI (Carbon Efficient Gain Index). This metric represents the carbon emission per unit percentage gain per million trainable parameters . This metric provides a normalized measure to compare model's efficiency in terms of performance improvement relative to their environmental cost. The experiment's outcome demonstrates that fine-tuning SLMs and VLMs can achieve performance levels comparable to Large Language Models (LLMs) while producing significantly less carbon emissions. Our findings suggest that the marginal gains in accuracy from larger models do not justify the substantial increase in carbon emissions. Leveraging lower-bit quantization levels, the proposed metric further enhances energy efficiency without compromising performance. This study highlights balancing high performance and environmental sustainability. It offers a valuable metric for selecting models suitable for environmentally-friendly AI development.

翻译：本文分析了小型语言模型（SLMs）和视觉语言模型（VLMs）的性能，并在图像描述、视觉问答（VQA）、对话摘要和文本到SQL转换这四项核心任务上，评估了模型性能与碳排放之间的权衡关系。研究选取了属于Qwen和LLaMA架构家族的多种SLMs与VLMs，并基于参数量、量化级别和微调参数等维度评估了不同模型变体。我们计算了各模型变体的性能及其碳排放量。为量化模型性能与碳排放之间的权衡，本文提出了一种新颖的指标——碳效率增益指数（CEGI）。该指标表示每百万可训练参数的单位百分比性能提升所对应的碳排放量。CEGI提供了一个标准化度量，用于比较模型在性能提升相对于环境成本方面的效率。实验结果表明，通过对SLMs和VLMs进行微调，可以达到与大型语言模型（LLMs）相当的性能水平，同时产生显著更低的碳排放。我们的发现表明，更大模型带来的精度边际提升并不能证明其碳排放的大幅增加是合理的。通过利用低位数量化级别，所提出的指标在保持性能的同时进一步提升了能源效率。本研究强调了在高性能与环境可持续性之间取得平衡的重要性，并为选择适合环境友好型人工智能开发的模型提供了一个有价值的度量标准。