Despite the successes of recent developments in visual AI, different shortcomings still exist; from missing exact logical reasoning, to abstract generalization abilities, to understanding complex and noisy scenes. Unfortunately, existing benchmarks, were not designed to capture more than a few of these aspects. Whereas deep learning datasets focus on visually complex data but simple visual reasoning tasks, inductive logic datasets involve complex logical learning tasks, however, lack the visual component. To address this, we propose the diagnostic visual logical learning dataset, V-LoL, that seamlessly combines visual and logical challenges. Notably, we introduce the first instantiation of V-LoL, V-LoL-Train, - a visual rendition of a classic benchmark in symbolic AI, the Michalski train problem. By incorporating intricate visual scenes and flexible logical reasoning tasks within a versatile framework, V-LoL-Train provides a platform for investigating a wide range of visual logical learning challenges. We evaluate a variety of AI systems including traditional symbolic AI, neural AI, as well as neuro-symbolic AI. Our evaluations demonstrate that even SOTA AI faces difficulties in dealing with visual logical learning challenges, highlighting unique advantages and limitations of each methodology. Overall, V-LoL opens up new avenues for understanding and enhancing current abilities in visual logical learning for AI systems.

翻译：尽管视觉人工智能领域的最新进展取得了诸多成功，但依然存在不同的缺陷：从缺乏精确的逻辑推理能力，到抽象泛化能力不足，再到对复杂嘈杂场景的理解有限。遗憾的是，现有基准数据集的设计未能同时涵盖这些方面的多个维度。深度学习数据集侧重于视觉复杂度高但推理任务简单的数据，而归纳逻辑数据集虽然涉及复杂的逻辑学习任务，却缺乏视觉成分。为解决这一问题，我们提出了诊断性视觉逻辑学习数据集V-LoL，它无缝融合了视觉与逻辑的双重挑战。特别地，我们首次实现了V-LoL的具体实例化——V-LoL-Train，该数据集是对符号人工智能经典基准问题（即Michalski火车问题）的视觉化重构。通过在一个灵活框架中融入复杂的视觉场景与多样化的逻辑推理任务，V-LoL-Train为研究广泛的视觉逻辑学习挑战提供了平台。我们评估了包括传统符号AI、神经AI以及神经符号AI在内的多种人工智能系统。实验结果表明，即使是当前最先进的AI系统在处理视觉逻辑学习挑战时仍面临困难，这凸显了各类方法独特的优势与局限性。总体而言，V-LoL为理解和提升AI系统在视觉逻辑学习方面的现有能力开辟了新途径。