Studies conducted on Western, Educated, Industrialized, Rich, and Democratic (WEIRD) samples are considered atypical of the world's population and may not accurately represent human behavior. In this study, we aim to quantify the extent to which the ACM FAccT conference, the leading venue in exploring Artificial Intelligence (AI) systems' fairness, accountability, and transparency, relies on WEIRD samples. We collected and analyzed 128 papers published between 2018 and 2022, accounting for 30.8% of the overall proceedings published at FAccT in those years (excluding abstracts, tutorials, and papers without human-subject studies or clear country attribution for the participants). We found that 84% of the analyzed papers were exclusively based on participants from Western countries, particularly exclusively from the U.S. (63%). Only researchers who undertook the effort to collect data about local participants through interviews or surveys added diversity to an otherwise U.S.-centric view of science. Therefore, we suggest that researchers collect data from under-represented populations to obtain an inclusive worldview. To achieve this goal, scientific communities should champion data collection from such populations and enforce transparent reporting of data biases.
翻译:针对西方化、受教育、工业化、富裕且民主(WEIRD)样本开展的研究被认为不能代表全球人口,且可能无法准确反映人类行为。本研究旨在量化ACM FAccT会议——这一人工智能系统公平性、问责制与透明性领域的顶级学术平台——对WEIRD样本的依赖程度。我们收集并分析了2018年至2022年间发表的128篇论文,占该期间FAccT会议全部论文集(不含摘要、教程以及未涉及人类受试者研究或未明确标注参与者国别的论文)的30.8%。研究发现,84%的被分析论文完全基于西方国家的参与者,其中63%完全仅基于美国参与者。唯有研究者通过访谈或问卷调查的方式收集本地参与者数据时,才为原本以美国为中心的科学视角增添了多样性。因此,我们建议研究者从代表性不足的人群中收集数据,以获得包容性的全球视野。为实现这一目标,科学界应倡导从这类人群收集数据,并强制要求对数据偏见进行透明报告。