BACKGROUND: Software Process Improvement (SPI) is a systematic approach to increase the efficiency and effectiveness of a software development organization and to enhance software products. OBJECTIVE: This paper aims to identify and characterize evaluation strategies and measurements used to assess the impact of different SPI initiatives. METHOD: The systematic literature review includes 148 papers published between 1991 and 2008. The selected papers were classified according to SPI initiative, applied evaluation strategies, and measurement perspectives. Potential confounding factors interfering with the evaluation of the improvement effort were assessed. RESULTS: Seven distinct evaluation strategies were identified, wherein the most common one, "Pre-Post Comparison" was applied in 49 percent of the inspected papers. Quality was the most measured attribute (62 percent), followed by Cost (41 percent), and Schedule (18 percent). Looking at measurement perspectives, "Project" represents the majority with 66 percent. CONCLUSION: The evaluation validity of SPI initiatives is challenged by the scarce consideration of potential confounding factors, particularly given that "Pre-Post Comparison" was identified as the most common evaluation strategy, and the inaccurate descriptions of the evaluation context. Measurements to assess the short and mid-term impact of SPI initiatives prevail, whereas long-term measurements in terms of customer satisfaction and return on investment tend to be less used.

翻译：背景：软件过程改进（SPI）是一种旨在提升软件开发组织的效率与效能、并改善软件产品的系统化方法。目标：本文旨在识别并描述用于评估不同SPI举措影响的评价策略及度量指标。方法：本系统性文献综述涵盖了1991年至2008年间发表的148篇论文。根据SPI举措、所采用评价策略及度量角度对所选论文进行分类，同时评估了可能干扰改进效果评价的混杂因素。结果：共识别出七种不同的评价策略，其中最常见的“前后对比”方法应用于49%的审阅论文。质量是最常被度量的属性（62%），其次是成本（41%）和进度（18%）。从度量角度看，“项目”层面占比最高，达66%。结论：SPI举措的评价有效性受到潜在混杂因素考虑不足的挑战，尤其在“前后对比”被确定为最常见的评价策略且评价情境描述不准确的情况下。当前普遍采用评估SPI举措短期及中期影响的度量指标，而面向客户满意度及投资回报的长期度量指标则应用较少。