Spatial reasoning tasks in multi-agent environments such as event prediction, agent type identification, or missing data imputation are important for multiple applications (e.g., autonomous surveillance over sensor networks and subtasks for reinforcement learning (RL)). StarCraft II game replays encode intelligent (and adversarial) multi-agent behavior and could provide a testbed for these tasks; however, extracting simple and standardized representations for prototyping these tasks is laborious and hinders reproducibility. In contrast, MNIST and CIFAR10, despite their extreme simplicity, have enabled rapid prototyping and reproducibility of ML methods. Following the simplicity of these datasets, we construct a benchmark spatial reasoning dataset based on StarCraft II replays that exhibit complex multi-agent behaviors, while still being as easy to use as MNIST and CIFAR10. Specifically, we carefully summarize a window of 255 consecutive game states to create 3.6 million summary images from 60,000 replays, including all relevant metadata such as game outcome and player races. We develop three formats of decreasing complexity: Hyperspectral images that include one channel for every unit type (similar to multispectral geospatial images), RGB images that mimic CIFAR10, and grayscale images that mimic MNIST. We show how this dataset can be used for prototyping spatial reasoning methods. All datasets, code for extraction, and code for dataset loading can be found at https://starcraftdata.davidinouye.com

翻译：多智能体环境中的空间推理任务（如事件预测、智能体类型识别或缺失数据填补）对多项应用至关重要（例如基于传感器网络的自主监控及强化学习子任务）。星际争霸II游戏回放编码了智能（且对抗性的）多智能体行为，可为这些任务提供测试平台；然而，为这些任务的原型设计提取简单且标准化的表示需要大量人力，且阻碍了可复现性。相比之下，MNIST与CIFAR10尽管极端简单，却实现了机器学习方法的快速原型设计与可复现性。遵循这些数据集的简洁性，我们基于展现复杂多智能体行为的星际争霸II回放构建了一个基准空间推理数据集，同时保持与MNIST及CIFAR10同等易用性。具体而言，我们精心汇总了255个连续游戏状态的滑动窗口，从6万场回放中生成360万张汇总图像，并附带所有相关元数据（如游戏结果与玩家种族）。我们开发了三种复杂度递减的格式：包含每种单位类型独立通道的高光谱图像（类似于多光谱地理空间图像）、模仿CIFAR10的RGB图像，以及模仿MNIST的灰度图像。我们展示了该数据集如何用于空间推理方法的原型设计。所有数据集、提取代码及数据集加载代码均可在https://starcraftdata.davidinouye.com 获取。