Omni-modal models that have multimodal input and output are emerging. However, benchmarking their multimodal generation, especially in image generation, is challenging due to the subtleties of human preferences and model biases. Many image generation benchmarks focus on aesthetics instead of the fine-grained generation capabilities of these models, failing to evaluate their visual intelligence with objective metrics. In PixelArena, we propose using semantic segmentation tasks to objectively examine their fine-grained generative intelligence with pixel precision. With our benchmark and experiments, we find the latest Gemini 3 Pro Image has emergent image generation capabilities that generate semantic masks with high fidelity under zero-shot settings, showcasing visual intelligence unseen before and true generalization in new image generation tasks. We further investigate its results, compare them qualitatively and quantitatively with those of other models, and present failure cases. The findings not only signal exciting progress in the field but also provide insights into future research related to dataset development, omni-modal model development, and the design of metrics.

翻译：具备多模态输入与输出的全模态模型正在兴起。然而，由于人类偏好的微妙性和模型偏差，对其多模态生成能力（尤其是图像生成）进行基准测试颇具挑战。许多图像生成基准侧重于美学而非模型的细粒度生成能力，无法通过客观指标评估其视觉智能。在PixelArena中，我们提出利用语义分割任务，以像素级精度客观检验模型的细粒度生成智能。通过我们的基准测试与实验，我们发现最新的Gemini 3 Pro Image展现出新兴的图像生成能力，能够在零样本设置下生成高保真度的语义掩码，这展示了前所未有的视觉智能以及在新图像生成任务中真正的泛化能力。我们进一步分析了其生成结果，与其他模型进行了定性与定量比较，并展示了失败案例。这些发现不仅标志着该领域令人兴奋的进展，也为未来与数据集开发、全模态模型构建及度量标准设计相关的研究提供了启示。