Software Engineering activities are information intensive. Research proposes Information Retrieval (IR) techniques to support engineers in their daily tasks, such as establishing and maintaining traceability links, fault identification, and software maintenance. We describe an engineering task, test case selection, and illustrate our problem analysis and solution discovery process. The objective of the study is to gain an understanding of to what extent IR techniques (one potential solution) can be applied to test case selection and provide decision support in a large-scale, industrial setting. We analyze, in the context of the studied company, how test case selection is performed and design a series of experiments evaluating the performance of different IR techniques. Each experiment provides lessons learned from implementation, execution, and results, feeding to its successor. The three experiments led to the following observations: 1) there is a lack of research on scalable parameter optimization of IR techniques for software engineering problems; 2) scaling IR techniques to industry data is challenging, in particular for latent semantic analysis; 3) the IR context poses constraints on the empirical evaluation of IR techniques, requiring more research on developing valid statistical approaches. We believe that our experiences in conducting a series of IR experiments with industry grade data are valuable for peer researchers so that they can avoid the pitfalls that we have encountered. Furthermore, we identified challenges that need to be addressed in order to bridge the gap between laboratory IR experiments and real applications of IR in the industry.

翻译：软件工程活动具有信息密集型特征。研究界提出应用信息检索技术支持工程师日常任务，包括建立与维护可追溯性链接、故障识别及软件维护等。本文描述了一项工程任务——测试用例选择，并阐述了问题分析与解决方案发现过程。本研究旨在理解信息检索技术在工业级大规模测试用例选择中的适用程度及其决策支持能力。我们结合所研究企业的实际场景，分析了测试用例选择的现有流程，设计了一系列实验评估不同信息检索技术的性能。每项实验均呈现实施、执行及结果中获得的经验教训，并为后续实验提供参考。三项实验得出以下发现：1）针对软件工程问题，信息检索技术的可扩展参数优化研究存在空白；2）将信息检索技术扩展至工业数据具有挑战性，尤其对于潜在语义分析而言；3）信息检索语境对其实证评估施加了约束条件，亟需开发有效统计方法的相关研究。我们相信，基于工业级数据开展系列信息检索实验的实践经验，对同行研究者避免重蹈覆辙具有重要价值。此外，我们识别出弥合实验室信息检索实验与工业实际应用之间鸿沟所亟需解决的关键挑战。