Large language models (LLMs) are gaining increasing popularity in both academia and industry, owing to their unprecedented performance in various applications. As LLMs continue to play a vital role in both research and daily use, their evaluation becomes increasingly critical, not only at the task level, but also at the society level for better understanding of their potential risks. Over the past years, significant efforts have been made to examine LLMs from various perspectives. This paper presents a comprehensive review of these evaluation methods for LLMs, focusing on three key dimensions: what to evaluate, where to evaluate, and how to evaluate. Firstly, we provide an overview from the perspective of evaluation tasks, encompassing general natural language processing tasks, reasoning, medical usage, ethics, educations, natural and social sciences, agent applications, and other areas. Secondly, we answer the `where' and `how' questions by diving into the evaluation methods and benchmarks, which serve as crucial components in assessing performance of LLMs. Then, we summarize the success and failure cases of LLMs in different tasks. Finally, we shed light on several future challenges that lie ahead in LLMs evaluation. Our aim is to offer invaluable insights to researchers in the realm of LLMs evaluation, thereby aiding the development of more proficient LLMs. Our key point is that evaluation should be treated as an essential discipline to better assist the development of LLMs. We consistently maintain the related open-source materials at: https://github.com/MLGroupJLU/LLM-eval-survey.

翻译：大语言模型因其在各类应用中的空前性能，在学术界和工业界日益受到青睐。随着大语言模型在研究与日常应用中持续发挥关键作用，其评估变得愈发重要——不仅涉及任务层面，更需从社会层面深入理解其潜在风险。近年来，学界已从多角度对大语言模型展开大量研究。本文系统梳理了针对大语言模型的现有评估方法，聚焦三大核心维度：评估什么、在哪里评估以及如何评估。首先，我们从评估任务视角进行概述，涵盖通用自然语言处理任务、推理、医疗应用、伦理、教育、自然科学与社会科学、智能体应用等领域。其次，通过深入分析评估方法与基准测试——这两者作为评估大语言模型性能的关键要素——解答了"在哪里"与"如何"评估的问题。接着，我们总结大语言模型在不同任务中的成功案例与失败案例。最后，我们展望了大语言模型评估领域面临的若干未来挑战。本文旨在为大语言模型评估研究者提供宝贵见解，从而助力开发更高效的大语言模型。我们的核心观点是：评估应作为一门关键学科，更好地支撑大语言模型的发展。相关开源资料持续维护于：https://github.com/MLGroupJLU/LLM-eval-survey。