We present WinSyn, a dataset consisting of high-resolution photographs and renderings of 3D models as a testbed for synthetic-to-real research. The dataset consists of 75,739 high-resolution photographs of building windows, including traditional and modern designs, captured globally. These include 89,318 cropped subimages of windows, of which 9,002 are semantically labeled. Further, we present our domain-matched photorealistic procedural model which enables experimentation over a variety of parameter distributions and engineering approaches. Our procedural model provides a second corresponding dataset of 21,290 synthetic images. This jointly developed dataset is designed to facilitate research in the field of synthetic-to-real learning and synthetic data generation. WinSyn allows experimentation into the factors that make it challenging for synthetic data to compete with real-world data. We perform ablations using our synthetic model to identify the salient rendering, materials, and geometric factors pertinent to accuracy within the labeling task. We chose windows as a benchmark because they exhibit a large variability of geometry and materials in their design, making them ideal to study synthetic data generation in a constrained setting. We argue that the dataset is a crucial step to enable future research in synthetic data generation for deep learning.

翻译：我们提出WinSyn数据集，该数据集包含高分辨率照片与三维模型渲染图，作为合成到现实研究的测试平台。该数据集包含全球采集的75,739张建筑窗户高分辨率照片，涵盖传统与现代设计风格，其中包括89,318张裁剪后的窗户子图像，其中9,002张已进行语义标注。此外，我们提出了领域匹配的光照真实感过程化模型，该模型支持在不同参数分布和工程方法下进行实验。我们的过程化模型提供了由21,290张合成图像组成的第二个对应数据集。这个联合开发的数据集旨在促进合成到现实学习和合成数据生成领域的研究。通过WinSyn，我们可对影响合成数据与真实世界数据竞争力的因素进行实验。我们利用合成模型进行消融研究，以确定与标注任务精度相关的关键渲染、材质和几何因素。选择窗户作为基准测试对象，是因为其设计在几何形态和材质方面具有高度多样性，使其成为在受限条件下研究合成数据生成的理想对象。我们认为该数据集是推动未来深度学习合成数据生成研究的关键一步。