As video games evolve into expansive, detailed worlds, visual quality becomes essential, yet increasingly challenging. Traditional testing methods, limited by resources, face difficulties in addressing the plethora of potential bugs. Machine learning offers scalable solutions; however, heavy reliance on large labeled datasets remains a constraint. Addressing this challenge, we propose a novel method, utilizing unlabeled gameplay and domain-specific augmentations to generate datasets & self-supervised objectives used during pre-training or multi-task settings for downstream visual bug detection. Our methodology uses weak-supervision to scale datasets for the crafted objectives and facilitates both autonomous and interactive weak-supervision, incorporating unsupervised clustering and/or an interactive approach based on text and geometric prompts. We demonstrate on first-person player clipping/collision bugs (FPPC) within the expansive Giantmap game world, that our approach is very effective, improving over a strong supervised baseline in a practical, very low-prevalence, low data regime (0.336 $\rightarrow$ 0.550 F1 score). With just 5 labeled "good" exemplars (i.e., 0 bugs), our self-supervised objective alone captures enough signal to outperform the low-labeled supervised settings. Building on large-pretrained vision models, our approach is adaptable across various visual bugs. Our results suggest applicability in curating datasets for broader image and video tasks within video games beyond visual bugs.

翻译：随着电子游戏发展为广阔而精细的虚拟世界，视觉质量变得至关重要，但也日益具有挑战性。受限于资源的传统测试方法难以应对大量潜在缺陷。机器学习提供可扩展的解决方案，但对大规模标注数据集的严重依赖仍是一个制约因素。为解决这一挑战，我们提出一种新颖方法，利用无标签游戏过程和领域特定增广来生成数据集及自监督目标，用于预训练或多任务设置中的下游视觉缺陷检测。我们的方法采用弱监督来扩展针对这些定制目标的数据集，并支持自主式和交互式弱监督，结合无监督聚类和/或基于文本与几何提示的交互方式。我们在广阔Giantmap游戏世界中的第一人称玩家碰撞/穿模缺陷上证明，我们的方法非常有效，在低标签率、缺陷稀疏的实际场景中，相较强监督基线取得显著提升（F1得分从0.336提升至0.550）。仅用5个标注“正常”样本（即无缺陷），我们的自监督目标便能捕获足够信号，超越低标签监督设置。基于大规模预训练视觉模型，我们的方法可适应多种视觉缺陷。结果表明，该方法在游戏领域除视觉缺陷外，还可推广至更广泛的图像与视频任务数据集构建。