Recent Large Language Models (LLMs) have demonstrated remarkable proficiency in code generation. However, their ability to create complex visualizations for scaled and structured data remains largely unevaluated and underdeveloped. To address this gap, we introduce PlotCraft, a new benchmark featuring 1k challenging visualization tasks that cover a wide range of topics, such as finance, scientific research, and sociology. The benchmark is structured around seven high-level visualization tasks and encompasses 48 distinct chart types. Crucially, it is the first to systematically evaluate both single-turn generation and multi-turn refinement across a diverse spectrum of task complexities. Our comprehensive evaluation of 23 leading LLMs on PlotCraft reveals obvious performance deficiencies in handling sophisticated visualization tasks. To bridge this performance gap, we develope SynthVis-30K, a large-scale, high-quality dataset of complex visualization code synthesized via a collaborative agent framework. Building upon this dataset, we develope PlotCraftor, a novel code generation model that achieves strong capabilities in complex data visualization with a remarkably small size. Across VisEval, PandasPlotBench, and our proposed PlotCraft, PlotCraftor shows performance comparable to that of leading proprietary approaches. Especially, on hard task, Our model achieves over 50% performance improvement. We will release the benchmark, dataset, and code at https://github.com/Speakn0w/PlotCraft-Benchmark.

翻译：近年来，大型语言模型（LLMs）在代码生成方面展现出卓越的能力。然而，其在处理规模化与结构化数据时创建复杂可视化的潜力尚未得到充分评估与开发。为填补这一空白，我们提出了PlotCraft——一个包含1000项高难度可视化任务的新基准测试集，涵盖金融、科学研究、社会学等多个领域。该基准围绕七类高层级可视化任务构建，包含48种不同的图表类型。关键的是，这是首个系统化评估单轮生成与多轮优化在多样化任务复杂度下表现的基准。我们在PlotCraft上对23个领先LLMs进行的全面评估显示，现有模型在处理复杂可视化任务时存在明显性能缺陷。为弥合这一差距，我们开发了SynthVis-30K——一个通过协作智能体框架合成的大规模高质量复杂可视化代码数据集。基于此数据集，我们构建了PlotCraftor，一个在模型体积极小的前提下实现强大复杂数据可视化能力的新型代码生成模型。在VisEval、PandasPlotBench及我们提出的PlotCraft基准测试中，PlotCraftor展现出与领先专有模型相媲美的性能。尤其在困难任务上，我们的模型实现了超过50%的性能提升。我们将通过https://github.com/Speakn0w/PlotCraft-Benchmark 公开基准测试集、数据集及代码。