Despite rapid progress on coding agents, progress on their multimodal counterparts has lagged behind. A key challenge is the scarcity of evaluation testbeds that combine the complexity of software development with the need for deep multimodal understanding. Game development provides such a testbed as agents must navigate large, dense codebases while manipulating intrinsically multimodal assets such as shaders, sprites, and animations within a visual game scene. We present GameDevBench, the first benchmark for evaluating agents on game development tasks. GameDevBench consists of 132 tasks derived from web and video tutorials. Tasks require significant multimodal understanding and are complex -- the average solution requires over three times the amount of lines of code and file changes compared to prior software development benchmarks. Agents still struggle with game development, with the best agent solving only 54.5% of tasks. We find a strong correlation between perceived task difficulty and multimodal complexity, with success rates dropping from 46.9% on gameplay-oriented tasks to 31.6% on 2D graphics tasks. To improve multimodal capability, we introduce two simple image and video-based feedback mechanisms for agents. Despite their simplicity, these methods consistently improve performance, with the largest change being an increase in Claude Sonnet 4.5's performance from 33.3% to 47.7%. We release GameDevBench publicly to support further research into agentic game development.

翻译：尽管编码智能体取得了快速进展，但其多模态对应体的发展却相对滞后。一个关键挑战在于缺乏能够将软件开发复杂性与深度多模态理解需求相结合的评价测试平台。游戏开发为此类测试提供了理想场景，因为智能体必须在视觉游戏场景中导航庞大而密集的代码库，同时操作着色器、精灵图、动画等本质多模态的资产。我们提出了GameDevBench——首个面向游戏开发任务的智能体评估基准。该基准包含132项源自网络及视频教程的任务，这些任务要求显著的多模态理解能力且复杂度高：平均解决方案所需的代码行数与文件修改量是先前软件开发基准的三倍以上。当前智能体在游戏开发任务中仍面临困难，最佳智能体仅能完成54.5%的任务。我们发现任务感知难度与多模态复杂度存在强相关性：在游戏玩法导向任务中成功率为46.9%，而在2D图形任务中则降至31.6%。为提升多模态能力，我们为智能体引入了两种基于图像和视频的简易反馈机制。尽管方法简单，这些机制能持续提升性能，其中Claude Sonnet 4.5的性能提升最为显著——从33.3%提高至47.7%。我们公开发布GameDevBench以支持智能体游戏开发领域的进一步研究。