While self-supervised pretraining has reduced vision systems' reliance on synthetic data, simulation remains an indispensable tool for closed-loop optimization and rigorous out-of-distribution (OOD) evaluation. However, modern simulation platforms often present steep technical barriers, requiring extensive expertise in computer graphics and game development. In this work, we present LychSim, a highly controllable and interactive simulation framework built upon Unreal Engine 5 to bridge this gap. LychSim is built around three key designs: (1) a streamlined Python API that abstracts away underlying engine complexities; (2) a procedural data pipeline capable of generating diverse, high-fidelity environments with varying out-of-distribution (OOD) visual challenges, paired with rich 2D and 3D ground truths; and (3) a native integration of the Model Context Protocol (MCP) that transforms the simulator into a dynamic, closed-loop playground for reasoning agentic LLMs. We further annotate scene-level procedural rules and object-level pose alignments to enable semantically aligned 3D ground truths and automated scene modification. We demonstrate LychSim's capability across multiple downstream applications, including serving as a synthetic data engine, powering reinforcement learning-based adversarial examiners, and facilitating interactive, language-driven scene layout generation. To benefit the broader vision community, LychSim will be made publicly available, including full source code and various data annotations.

翻译：尽管自监督预训练已降低了视觉系统对合成数据的依赖，但仿真仍是闭环优化与严格分布外（OOD）评估不可或缺的工具。然而，现有仿真平台往往存在陡峭的技术门槛，要求使用者具备计算机图形学与游戏开发领域的深厚专业知识。本文提出LychSim——一个基于虚幻引擎5构建的高可控交互式仿真框架，以弥合这一鸿沟。LychSim围绕三项核心设计构建：（1）精简的Python接口，屏蔽底层引擎的复杂性；（2）程序化数据管线，可生成带有不同分布外（OOD）视觉挑战的多样化高保真环境，并配套丰富的2D/3D真值；及（3）原生集成模型上下文协议（MCP），将仿真器转变为推理型智能体LLM的动态闭环实验场。我们进一步标注了场景级程序化规则与对象级位姿对齐，以实现语义对齐的3D真值与自动化场景修改。我们通过多项下游应用展示了LychSim的能力：作为合成数据引擎、赋能基于强化学习的对抗性评估器、以及支持交互式语言驱动的场景布局生成。为惠及更广泛的视觉社区，LychSim将公开提供，包括完整源代码与多种数据标注。