With the continuous emergence of Chinese Large Language Models (LLMs), how to evaluate a model's capabilities has become an increasingly significant issue. The absence of a comprehensive Chinese benchmark that thoroughly assesses a model's performance, the unstandardized and incomparable prompting procedure, and the prevalent risk of contamination pose major challenges in the current evaluation of Chinese LLMs. We present CLEVA, a user-friendly platform crafted to holistically evaluate Chinese LLMs. Our platform employs a standardized workflow to assess LLMs' performance across various dimensions, regularly updating a competitive leaderboard. To alleviate contamination, CLEVA curates a significant proportion of new data and develops a sampling strategy that guarantees a unique subset for each leaderboard round. Empowered by an easy-to-use interface that requires just a few mouse clicks and a model API, users can conduct a thorough evaluation with minimal coding. Large-scale experiments featuring 23 influential Chinese LLMs have validated CLEVA's efficacy.

翻译：随着中文大语言模型（LLMs）的持续涌现，如何评估模型能力已成为日益重要的问题。当前中文LLMs评估面临三大主要挑战：缺乏全面评估模型性能的中文基准、不统一且不可比较的提示流程，以及普遍存在的污染风险。我们提出CLEVA，一个专为全面评估中文LLMs而设计的用户友好型平台。该平台采用标准化工作流程，从多个维度评估LLMs性能，并定期更新具有竞争力的排行榜。为缓解污染问题，CLEVA专门收集了大量新数据，并开发了采样策略以确保每轮排行榜使用独特的子集。借助只需点击几下鼠标和模型API的简易界面，用户可用极少的代码完成全面评估。涵盖23个有影响力的中文LLMs的大规模实验验证了CLEVA的有效性。