Recent years have seen impressive advances in text-to-image generation, with image generative or unified models producing high-quality images from text. Yet these models still struggle with fine-grained color controllability, often failing to accurately match colors specified in text prompts. While existing benchmarks evaluate compositional reasoning and prompt adherence, none systematically assess color precision. Color is fundamental to human visual perception and communication, critical for applications from art to design workflows requiring brand consistency. However, current benchmarks either neglect color or rely on coarse assessments, missing key capabilities such as interpreting RGB values or aligning with human expectations. To this end, we propose GenColorBench, the first comprehensive benchmark for text-to-image color generation, grounded in color systems like ISCC-NBS and CSS3/X11, including numerical colors which are absent elsewhere. With 44K color-focused prompts covering 400+ colors, it reveals models' true capabilities via perceptual and automated assessments. Evaluations of popular text-to-image models using GenColorBench show performance variations, highlighting which color conventions models understand best and identifying failure modes. Our GenColorBench assessments will guide improvements in precise color generation. The benchmark will be made public upon acceptance.

翻译：近年来，文本到图像生成技术取得了令人瞩目的进展，图像生成模型或统一模型能够根据文本生成高质量图像。然而，这些模型在细粒度色彩可控性方面仍存在困难，往往无法准确匹配文本提示中指定的颜色。现有基准主要评估组合推理和提示遵循能力，尚未系统性地评估色彩精度。色彩是人类视觉感知和交流的基础，对于从艺术到需要品牌一致性的设计工作流等应用至关重要。然而，当前基准要么忽视色彩评估，要么依赖粗略评估方法，缺失了关键能力评估，如对RGB值的解释或与人类期望的对齐。为此，我们提出GenColorBench——首个基于ISCC-NBS和CSS3/X11等色彩系统的文本到图像色彩生成综合基准，其中包含其他基准缺失的数值化色彩描述。通过涵盖400余种颜色的4.4万个色彩中心提示，该基准通过感知评估和自动化评估揭示了模型的真实能力。使用GenColorBench对主流文本到图像模型的评估显示了性能差异，明确了模型最擅长的色彩约定并识别了失败模式。我们的GenColorBench评估将为精确色彩生成的改进提供指导。该基准将在论文录用后公开。