The most frequently used method to collect research data online is crowdsouring and its use continues to grow rapidly. This report investigates for the first time whether researchers also have to expect significantly different hardware performance when deploying to Amazon Mechanical Turk (MTurk). This is assessed by collecting basic hardware parameters (Operating System, GPU, and used browser) from Amazon Mechanical Turk (MTurk) and a traditional recruitment method (i.e., snowballing). The significant hardware differences between crowdsourcing participants (MTurk) and snowball recruiting are reported including relevant descriptive statistics for assessing hardware performance of 3D web applications. The report suggests that hardware differences need to be considered to obtain valid results if the designed experiment application requires graphical intense computations and relies on a coherent user experience of MTurk and more established recruitment strategies (i.e. snowballing).
翻译:在线研究数据最常用的收集方法是众包,且其使用规模持续快速增长。本报告首次探究了研究人员在通过Amazon Mechanical Turk(MTurk)平台部署实验时,是否也需要应对显著不同的硬件性能。通过从Amazon Mechanical Turk(MTurk)和传统招募方法(即滚雪球抽样)收集基础硬件参数(操作系统、GPU及使用的浏览器),评估了两类方法的硬件差异。报告揭示了众包参与者(MTurk)与滚雪球招募之间显著的硬件差异,并提供了用于评估3D网页应用硬件性能的相关描述性统计。研究表明,若设计的实验应用需密集的图形计算,且依赖MTurk与更成熟的招募策略(即滚雪球抽样)保持连贯的用户体验,则必须考虑硬件差异,以获得有效的研究结果。