In human factor fields such as human-computer interaction (HCI), psychology, and behavioral sciences, researchers have been concerned that participant samples are skewed toward WEIRD, i.e., participants mostly come from Western, Educated, Industrialized, Rich, and Democratic societies. This WEIRD skew may affect the generalizability of study results and hinder understanding of diverse participant populations and their cultural differences. The usable privacy and security (UPS) field has inherited many research methodologies from research on human factor fields such as HCI. We conducted a literature review to understand the extent to which participant samples in UPS papers were WEIRD and the characteristics of the methodologies and research topics in each user study recruiting Western or non-Western participants. We found that the skew toward WEIRD in UPS is greater than that in HCI. Geographic and linguistic barriers in the study methods and recruitment methods may cause researchers to conduct a user study locally. In addition, many papers did not report participant demographics, which could hinder the replication of the reported studies, leading to low reproducibility. We provide the following suggestions to improve geographic diversity: facilitate replication studies, improve reproducibility, address issues of study and recruiting methods, diversify researchers, and facilitate research on the topics for non-WEIRD populations.
翻译:在人类因素领域如人机交互、心理学和行为科学中,研究者一直担忧参与者样本偏向于WEIRD群体,即参与者大多来自西方、教育程度高、工业化、富裕和民主社会。这种WEIRD偏向可能影响研究结果的普适性,并阻碍对多样化参与者群体及其文化差异的理解。可用隐私与安全领域继承了人机交互等人类因素领域的许多研究方法。我们通过文献综述来了解UPS论文中参与者样本的WEIRD程度,以及招募西方或非西方参与者的每项用户研究中方法论和研究主题的特征。我们发现UPS中的WEIRD偏向比HCI更为严重。研究方法与招募过程中的地理和语言障碍可能导致研究者仅在本地开展用户研究。此外,许多论文未报告参与者的人口统计信息,这可能阻碍所报告研究的复现,导致低可重复性。我们提出以下建议以改善地理多样性:促进复现研究、提高可重复性、解决研究与招募方法的问题、实现研究者多样化,并推动针对非WEIRD群体主题的研究。