Multimodal Large Language Models (MLLMs) have made significant strides in visual understanding and generation tasks. However, generating interleaved image-text content remains a challenge, which requires integrated multimodal understanding and generation abilities. While the progress in unified models offers new solutions, existing benchmarks are insufficient for evaluating these methods due to data size and diversity limitations. To bridge this gap, we introduce GATE OpenING (OpenING), a comprehensive benchmark comprising 5,400 high-quality human-annotated instances across 56 real-world tasks. OpenING covers diverse daily scenarios such as travel guide, design, and brainstorming, offering a robust platform for challenging interleaved generation methods. In addition, we present IntJudge, a judge model for evaluating open-ended multimodal generation methods. Trained with a novel data pipeline, our IntJudge achieves an agreement rate of 82. 42% with human judgments, outperforming GPT-based evaluators by 11.34%. Extensive experiments on OpenING reveal that current interleaved generation methods still have substantial room for improvement. Key findings on interleaved image-text generation are further presented to guide the development of next-generation models. The OpenING is open-sourced at https://opening-benchmark.github.io.

翻译：多模态大语言模型（MLLMs）在视觉理解与生成任务上已取得显著进展。然而，生成交错排列的图像-文本内容仍是一项挑战，这需要整合的多模态理解与生成能力。尽管统一模型的进展提供了新的解决方案，但现有基准由于数据规模和多样性的限制，不足以充分评估这些方法。为弥补这一差距，我们引入了GATE OpenING（OpenING），这是一个包含5,400个高质量人工标注实例、覆盖56个现实世界任务的综合性基准。OpenING涵盖了旅行指南、设计、头脑风暴等多种日常场景，为挑战交错生成方法提供了一个稳健的平台。此外，我们提出了IntJudge，一个用于评估开放式多模态生成方法的评判模型。通过新颖的数据流程训练，我们的IntJudge与人类判断的一致率达到82.42%，优于基于GPT的评估器11.34%。在OpenING上进行的大量实验表明，当前的交错生成方法仍有巨大的改进空间。我们进一步提出了关于交错图文生成的关键发现，以指导下一代模型的开发。OpenING已在https://opening-benchmark.github.io开源。