Amidst the rapid evolution of LLMs, the significance of evaluation in comprehending and propelling these models forward is increasingly paramount. Evaluations have revealed that factors such as scaling, training types, architectures and other factors profoundly impact the performance of LLMs. However, the extent and nature of these impacts continue to be subjects of debate because most assessments have been restricted to a limited number of models and data points. Clarifying the effects of these factors on performance scores can be more effectively achieved through a statistical lens. Our study embarks on a thorough re-examination of these LLMs, targeting the inadequacies in current evaluation methods. With the advent of a uniform evaluation framework, our research leverages an expansive dataset of evaluation results, introducing a comprehensive statistical methodology. This includes the application of ANOVA, Tukey HSD tests, GAMM, and clustering technique, offering a robust and transparent approach to deciphering LLM performance data. Contrary to prevailing findings, our results challenge assumptions about emergent abilities and the influence of given training types and architectures in LLMs. These findings furnish new perspectives on the characteristics, intrinsic nature, and developmental trajectories of LLMs. By providing straightforward and reliable methods to scrutinize and reassess LLM performance data, this study contributes a nuanced perspective on LLM efficiency and potentials.

翻译：随着LLMs的快速发展，评估在理解和推动这些模型进步中的重要性日益凸显。已有评估表明，规模扩展、训练类型、架构设计等因素对LLMs的性能产生深远影响。然而，由于现有评估大多局限于有限模型数量和样本数据，这些影响的具体程度与本质仍存争议。通过统计视角能更有效地厘清这些因素对性能指标的作用机制。本研究针对当前评估方法的局限性，对LLMs展开系统性再评估。借助统一评估框架的出现，我们利用大规模评估结果数据集，提出一套综合统计方法论。该方法整合了方差分析（ANOVA）、Tukey HSD检验、广义加性混合模型（GAMM）及聚类技术，为解析LLM性能数据提供了稳健透明的分析路径。与主流结论相异，我们的研究结果对LLMs中涌现能力的假设以及特定训练类型与架构的影响提出了挑战。这些发现为理解LLMs的特性、本质属性与发展轨迹提供了新视角。通过提供简洁可靠的方法来审视与再评估LLM性能数据，本研究为理解LLM的效能与潜力贡献了细致入微的见解。