In software-engineering research, many empirical studies are conducted with open-source or industry developers. However, in contrast to other research communities like economics or psychology, only few experiments use financial incentives (i.e., paying money) as a strategy to motivate participants' behavior and reward their performance. The most recent version of the SIGSOFT Empirical Standards mentions payouts only for increasing participation in surveys, but not for mimicking real-world motivations and behavior in experiments. Within this article, we report a controlled experiment in which we tackled this gap by studying how different financial incentivization schemes impact developers. For this purpose, we first conducted a survey on financial incentives used in the real-world, based on which we designed three incentivization schemes: (1) a performance-dependent scheme that employees prefer, (2) a scheme that is performance-independent, and (3) a scheme that mimics open-source development. Then, using a between-subject experimental design, we explored how these three schemes impact participants' performance. Our findings indicate that the different schemes can impact participants' performance in software-engineering experiments. Due to the small sample sizes, our results are not statistically significant, but we can still observe clear tendencies. Our contributions help understand the impact of financial incentives on participants in experiments as well as real-world scenarios, guiding researchers in designing experiments and organizations in compensating developers.
翻译:在软件工程研究中,许多实证研究是与开源或行业开发者合作进行的。然而,与经济学或心理学等其他研究领域相比,只有少数实验采用财务激励(即支付金钱)作为激励参与者行为和奖励其表现的策略。最新版SIGSOFT实证标准仅提及通过报酬来提高调查参与度,而未涉及在实验中模拟现实世界动机和行为。本文报告了一项对照实验,通过研究不同财务激励方案如何影响开发者来弥补这一空白。为此,我们首先开展了一项关于现实世界中财务激励使用情况的调查,并基于此设计了三种激励方案:(1)员工偏好的绩效依赖型方案,(2)绩效独立型方案,以及(3)模拟开源开发的方案。随后,采用组间实验设计,我们探究了这三种方案如何影响参与者的表现。研究结果表明,不同激励方案能够影响软件工程实验中参与者的表现。由于样本量较小,我们的结果未达到统计显著性,但仍可观察到明显的趋势。本研究的贡献在于帮助理解财务激励对实验参与者及现实场景的影响,为研究者设计实验以及组织制定开发者薪酬方案提供指导。