AI alignment refers to models acting towards human-intended goals, preferences, or ethical principles. Given that most large-scale deep learning models act as black boxes and cannot be manually controlled, analyzing the similarity between models and humans can be a proxy measure for ensuring AI safety. In this paper, we focus on the models' visual perception alignment with humans, further referred to as AI-human visual alignment. Specifically, we propose a new dataset for measuring AI-human visual alignment in terms of image classification, a fundamental task in machine perception. In order to evaluate AI-human visual alignment, a dataset should encompass samples with various scenarios that may arise in the real world and have gold human perception labels. Our dataset consists of three groups of samples, namely Must-Act (i.e., Must-Classify), Must-Abstain, and Uncertain, based on the quantity and clarity of visual information in an image and further divided into eight categories. All samples have a gold human perception label; even Uncertain (severely blurry) sample labels were obtained via crowd-sourcing. The validity of our dataset is verified by sampling theory, statistical theories related to survey design, and experts in the related fields. Using our dataset, we analyze the visual alignment and reliability of five popular visual perception models and seven abstention methods. Our code and data is available at \url{https://github.com/jiyounglee-0523/VisAlign}.

翻译：[translated abstract in Chinese] AI对齐指模型行为符合人类预期目标、偏好或伦理原则。鉴于大多数大规模深度学习模型是黑箱且无法手动控制，分析模型与人类之间的相似性可作为确保AI安全的代理度量。本文聚焦于模型与人类的视觉感知对齐（以下简称AI-人类视觉对齐），具体而言，我们提出一个新的数据集，用于衡量图像分类（机器感知的基础任务）中的AI-人类视觉对齐程度。为评估AI-人类视觉对齐，数据集需涵盖现实世界可能出现的各类场景样本，并具备金标准人类感知标签。本数据集基于图像中视觉信息的数量与清晰度，将样本分为三类——即必行动（即必分类）、必弃权与不确定，并进一步细分为八个子类。所有样本均具备金标准人类感知标签，其中不确定（严重模糊）样本的标签通过众包方式获取。数据集的效度通过抽样理论、调查设计相关统计理论及领域专家验证。利用该数据集，我们分析了五种主流视觉感知模型和七种弃权方法的视觉对齐程度与可靠性。我们的代码与数据公开于\url{https://github.com/jiyounglee-0523/VisAlign}。