This paper unveils CG-Eval, the first-ever comprehensive and automated evaluation framework designed for assessing the generative capabilities of large Chinese language models across a spectrum of academic disciplines. CG-Eval stands out for its automated process, which critically assesses models based on their proficiency in generating precise and contextually relevant responses to a diverse array of questions within six key domains: Science and Engineering, Humanities and Social Sciences, Mathematical Calculations, Medical Practitioner Qualification Examination, Judicial Examination, and Certified Public Accountant Examination. Alongside this, we introduce Gscore, an innovative composite index developed from a weighted sum of multiple metrics. Gscore uniquely automates the quality measurement of a model's text generation against reference standards, providing a detailed and nuanced assessment of model performance. This automation not only enhances the efficiency and scalability of the evaluation process but also ensures objective and consistent assessment across various models. The detailed test data and results, highlighting the robust capabilities and comparative performance of the evaluated models, are accessible at http://cgeval.besteasy.com/.

翻译：本文提出CG-Eval，这是首个面向跨学科学术领域、用于评估大型中文语言模型生成能力的全自动综合评估框架。CG-Eval的核心优势在于其自动化流程，能够从六大关键领域（科学与工程、人文社科、数学计算、医师资格考试、司法考试、注册会计师考试）的多维度问题中，严格评估模型生成精准且上下文关联回答的能力。我们同步引入Gscore这一创新复合指标，该指标基于多指标加权求和构建，可自动量化模型文本生成内容与参考标准之间的质量差异，提供细致且多维度的性能评估。该自动化机制不仅提升了评估流程的效率和可扩展性，还确保了跨模型评估的客观性与一致性。详细测试数据及结果（涵盖被评估模型的强大能力与对比表现）可通过 http://cgeval.besteasy.com/ 获取。