A recurring problem in software development is incorrect decision making on the techniques, methods and tools to be used. Mostly, these decisions are based on developers' perceptions about them. A factor influencing people's perceptions is past experience, but it is not the only one. In this research, we aim to discover how well the perceptions of the defect detection effectiveness of different techniques match their real effectiveness in the absence of prior experience. To do this, we conduct an empirical study plus a replication. During the original study, we conduct a controlled experiment with students applying two testing techniques and a code review technique. At the end of the experiment, they take a survey to find out which technique they perceive to be most effective. The results show that participants' perceptions are wrong and that this mismatch is costly in terms of quality. In order to gain further insight into the results, we replicate the controlled experiment and extend the survey to include questions about participants' opinions on the techniques and programs. The results of the replicated study confirm the findings of the original study and suggest that participants' perceptions might be based not on their opinions about complexity or preferences for techniques but on how well they think that they have applied the techniques.
翻译:软件开发中反复出现的问题是对所采用技术、方法和工具的错误决策。这些决策大多基于开发人员对其的感知。影响人们感知的因素之一是过往经验,但并非唯一因素。本研究旨在探究,在缺乏先验经验的情况下,不同技术的缺陷检测有效性的感知与其实际有效性之间的匹配程度。为此,我们开展了一项实证研究及重复实验。在原始研究中,我们通过受控实验让学生应用两种测试技术和一种代码审查技术。实验结束后,通过问卷调查确定他们认为最有效的技术。结果显示,参与者的感知存在偏差,且这种不匹配在质量方面代价高昂。为深入理解实验结果,我们重复了该受控实验,并扩展了问卷内容,添加了关于参与者对技术和程序看法的问题。重复实验的结果证实了原始研究的发现,并表明参与者的感知可能并非基于其对技术复杂性或偏好的判断,而是基于他们认为自己应用技术的熟练程度。