Large language models (LLMs) are gaining increasing popularity in both academia and industry, owing to their unprecedented performance in various applications. As LLMs continue to play a vital role in both research and daily use, their evaluation becomes increasingly critical, not only at the task level, but also at the society level for better understanding of their potential risks. Over the past years, significant efforts have been made to examine LLMs from various perspectives. This paper presents a comprehensive review of these evaluation methods for LLMs, focusing on three key dimensions: what to evaluate, where to evaluate, and how to evaluate. Firstly, we provide an overview from the perspective of evaluation tasks, encompassing general natural language processing tasks, reasoning, medical usage, ethics, educations, natural and social sciences, agent applications, and other areas. Secondly, we answer the `where' and `how' questions by diving into the evaluation methods and benchmarks, which serve as crucial components in assessing performance of LLMs. Then, we summarize the success and failure cases of LLMs in different tasks. Finally, we shed light on several future challenges that lie ahead in LLMs evaluation. Our aim is to offer invaluable insights to researchers in the realm of LLMs evaluation, thereby aiding the development of more proficient LLMs. Our key point is that evaluation should be treated as an essential discipline to better assist the development of LLMs. We consistently maintain the related open-source materials at: https://github.com/MLGroupJLU/LLM-eval-survey.

翻译：大型语言模型（LLMs）因其在各应用领域展现的前所未有的性能，正日益受到学术界和工业界的关注。随着LLMs在研究与日常使用中持续发挥关键作用，其评估变得愈发重要——不仅需要关注任务层面的表现，更需从社会层面理解其潜在风险。过去几年间，研究者已从多重视角对LLMs展开深入探究。本文系统梳理了现有LLM评估方法，聚焦三大关键维度：评估内容、评估场景与评估方式。首先，我们从评估任务视角进行概述，涵盖通用自然语言处理任务、推理能力、医疗应用、伦理问题、教育领域、自然科学与社会科学、智能体应用及其他方向。其次，通过深入剖析评估方法与基准测试（评估LLM性能的核心组件），我们解答了"评估场景"与"评估方式"这两个关键问题。随后，我们总结了LLMs在不同任务中的成功案例与失败教训。最后，展望了LLM评估领域面临的前沿挑战。本研究旨在为LLM评估领域的研究者提供宝贵启示，助力开发更高效的LLMs。我们的核心观点在于：评估应被视为一门支撑LLM发展的基础学科。相关开源资源持续维护于：https://github.com/MLGroupJLU/LLM-eval-survey。