In software-engineering research, many empirical studies are conducted with open-source or industry developers. However, in contrast to other research communities like economics or psychology, only few experiments use financial incentives (i.e., paying money) as a strategy to motivate participants' behavior and reward their performance. The most recent version of the SIGSOFT Empirical Standards mentions payouts only for increasing participation in surveys, but not for mimicking real-world motivations and behavior in experiments. Within this article, we report a controlled experiment in which we tackled this gap by studying how different financial incentivization schemes impact developers. For this purpose, we first conducted a survey on financial incentives used in the real-world, based on which we designed three incentivization schemes: (1) a performance-dependent scheme that employees prefer, (2) a scheme that is performance-independent, and (3) a scheme that mimics open-source development. Then, using a between-subject experimental design, we explored how these three schemes impact participants' performance. Our findings indicate that the different schemes can impact participants' performance in software-engineering experiments. Due to the small sample sizes, our results are not statistically significant, but we can still observe clear tendencies. Our contributions help understand the impact of financial incentives on participants in experiments as well as real-world scenarios, guiding researchers in designing experiments and organizations in compensating developers.
翻译:在软件工程研究中,许多实证研究是与开源或行业开发者共同开展的。然而,与经济学或心理学等其他研究领域不同,很少有实验采用财务激励(即支付金钱)作为激励参与者行为和奖励其表现的策略。最新的SIGSOFT实证标准仅在提高调查参与度方面提及报酬,但未涉及在实验中模拟真实世界的动机和行为。本文通过一项受控实验填补这一空白,研究不同财务激励方案如何影响开发者。为此,我们首先开展了一项关于现实世界中财务激励使用的调查,并基于此设计了三种激励方案:(1)员工偏好的绩效依赖型方案,(2)绩效独立型方案,以及(3)模拟开源开发的方案。随后,通过被试间实验设计,我们探究了这三种方案对参与者表现的影响。研究结果表明,不同方案可能影响参与者在软件工程实验中的表现。由于样本量较小,我们的结果未达到统计显著性,但仍能观察到明显趋势。我们的贡献有助于理解财务激励对实验参与者及现实场景中开发者的影响,为研究者设计实验以及组织补偿开发者提供指导。