AI alignment refers to models acting towards human-intended goals, preferences, or ethical principles. Given that most large-scale deep learning models act as black boxes and cannot be manually controlled, analyzing the similarity between models and humans can be a proxy measure for ensuring AI safety. In this paper, we focus on the models' visual perception alignment with humans, further referred to as AI-human visual alignment. Specifically, we propose a new dataset for measuring AI-human visual alignment in terms of image classification, a fundamental task in machine perception. In order to evaluate AI-human visual alignment, a dataset should encompass samples with various scenarios that may arise in the real world and have gold human perception labels. Our dataset consists of three groups of samples, namely Must-Act (i.e., Must-Classify), Must-Abstain, and Uncertain, based on the quantity and clarity of visual information in an image and further divided into eight categories. All samples have a gold human perception label; even Uncertain (severely blurry) sample labels were obtained via crowd-sourcing. The validity of our dataset is verified by sampling theory, statistical theories related to survey design, and experts in the related fields. Using our dataset, we analyze the visual alignment and reliability of five popular visual perception models and seven abstention methods. Our code and data is available at \url{https://github.com/jiyounglee-0523/VisAlign}.

翻译：AI对齐指的是模型按照人类预期的目标、偏好或伦理原则行动。鉴于大多数大规模深度学习模型以黑箱方式运作且无法手动控制，分析模型与人类之间的相似性可以作为确保AI安全性的代理度量。本文聚焦于模型与人类在视觉感知上的对齐，进一步称为AI-人类视觉对齐。具体而言，我们提出一个新的数据集，用于衡量图像分类（机器感知中的基础任务）方面的AI-人类视觉对齐程度。为了评估AI-人类视觉对齐，数据集应包含涵盖现实世界可能出现的各种场景的样本，并具备人类感知的金标准标签。我们的数据集根据图像中视觉信息的数量和清晰度，包含三组样本：必须行动（即必须分类）、必须弃权和不确定，并进一步划分为八个类别。所有样本均具有人类感知的金标准标签；即使是严重模糊的不确定样本标签也通过众包方式获取。我们数据集的有效性通过抽样理论、与调查设计相关的统计理论以及相关领域专家得到了验证。利用我们的数据集，我们分析了五种流行的视觉感知模型和七种弃权方法在视觉对齐和可靠性方面的表现。我们的代码和数据可在\url{https://github.com/jiyounglee-0523/VisAlign}获取。