Live programming features can be found in a range of programming environments, from individual prototypes to widely used environments. While liveness is generally considered a useful property, there is little empirical evidence on when and how liveness can be beneficial. Even though there are few experimental studies, their results are largely inconclusive. We reviewed existing experiments and related studies to gather a collection of potential effects of liveness and moderating factors. Based on this collection, we concluded that **task complexity** and **prior experience addressing liveness** are potentially essential factors neglected in previous experiments. To fill this gap, we devised and conducted a controlled experiment (N = 37) testing the hypothesis that task complexity moderates the effects of live introspection tools on participants? debugging efficiency, given participants with prior experience with liveness. Our results do not support the hypothesis that task complexity moderates the effect of live introspection tools. This non-significant moderation effect might result from the low number of participants, as the data suggests a moderation effect. The results also show that in our experiment setting, live introspection tools significantly reduced the time participants took to debug the tasks. For researchers interested in the effects of liveness, our findings suggest that studies on liveness should make conscious choices about task complexity and participants' prior experience with liveness. For designers of programming environments, the results of our experiment are a step toward understanding when and how programming tools should be live to become more helpful to programmers.
翻译:实时编程特性存在于从个人原型到广泛使用的各类编程环境中。尽管实时性通常被视为有益属性,但关于其何时以及如何产生效益的实证证据仍显不足。尽管已有少量实验研究,但其结论大多缺乏确定性。我们系统回顾了现有实验及相关研究,归纳出实时性可能产生的效应及其调节因素。基于此分析,我们得出结论:**任务复杂性**与**处理实时性的先验经验**可能是先前实验中被忽视的关键因素。为填补这一空白,我们设计并实施了受控实验(N = 37),在参与者具备实时性先验经验的条件下,检验"任务复杂性调节实时内省工具对调试效率影响"的假设。实验结果不支持任务复杂性调节实时内省工具效应的假设。这种不显著的调节效应可能源于样本量不足,因为数据趋势仍显示调节效应存在。结果同时表明,在本实验设置中,实时内省工具显著缩短了参与者调试任务所需时间。对于关注实时性效应的研究者,我们的发现建议相关研究应审慎考量任务复杂性和参与者实时性先验经验的设计选择。对于编程环境设计者,本实验结果为理解编程工具应在何时、以何种方式实现实时性以提升程序员效率提供了阶段性认知。