Code review is a critical practice in modern software engineering, helping developers detect defects early, improve code quality, and facilitate knowledge sharing. With the rapid advancement of large language models (LLMs), a growing body of work has explored automated support for code review. However, progress in this area is hindered by the lack of a systematic understanding of existing benchmarks and evaluation practices. Current code review datasets are scattered, vary widely in design, and provide limited insight into what review capabilities are actually being assessed. In this paper, we present a comprehensive survey of code review benchmarks spanning both the Pre-LLM and LLM eras (2015--2025). We analyze 99 research papers (58 Pre-LLM era and 41 LLM era) and extract key metadata, including datasets, evaluation metrics, data sources, and target tasks. Based on this analysis, we propose a multi-level taxonomy that organizes code review research into five domains and 18 fine-grained tasks. Our study reveals a clear shift toward end-to-end generative peer review, increasing multilingual coverage, and a decline in standalone change understanding tasks. We further identify limitations of current benchmarks and outline future directions, including broader task coverage, dynamic runtime evaluation, and taxonomy-guided fine-grained assessment. This survey provides a structured foundation for developing more realistic and comprehensive benchmarks for LLM-based code review.

翻译：代码审查是现代软件工程中的关键实践，有助于开发者及早发现缺陷、提升代码质量并促进知识共享。随着大语言模型（LLMs）的快速发展，越来越多的研究工作开始探索代码审查的自动化支持。然而，由于缺乏对现有基准与评估实践的系统性理解，该领域的进展受到阻碍。当前的代码审查数据集分散、设计差异巨大，且对实际评估的审查能力提供有限洞察。本文对跨越Pre-LLM与LLM时代（2015–2025）的代码审查基准进行了全面综述。我们分析了99篇研究论文（58篇Pre-LLM时代，41篇LLM时代），并提取了关键元数据，包括数据集、评估指标、数据来源及目标任务。基于此分析，我们提出了一个多层次分类法，将代码审查研究组织为五个领域和18项细粒度任务。我们的研究揭示了向端到端生成式同行评审的明显转变、多语言覆盖度的增加，以及独立变更理解任务的减少。我们进一步指出了当前基准的局限性，并展望了未来方向，包括更广泛的任务覆盖、动态运行时评估以及基于分类法的细粒度评估。本综述为开发更现实、更全面的基于LLM的代码审查基准提供了结构化基础。