Randomized experimental comparisons of alternative pedagogical strategies could provide useful empirical evidence in instructors' decision-making. However, traditional experiments do not have a clear and simple pathway to using data rapidly to try to increase the chances that students in an experiment get the best conditions. Drawing inspiration from the use of machine learning and experimentation in product development at leading technology companies, we explore how adaptive experimentation might help in continuous course improvement. In adaptive experiments, as different arms/conditions are deployed to students, data is analyzed and used to change the experience for future students. This can be done using machine learning algorithms to identify which actions are more promising for improving student experience or outcomes. This algorithm can then dynamically deploy the most effective conditions to future students, resulting in better support for students' needs. We illustrate the approach with a case study providing a side-by-side comparison of traditional and adaptive experimentation of self-explanation prompts in online homework problems in a CS1 course. This provides a first step in exploring the future of how this methodology can be useful in bridging research and practice in doing continuous improvement.

翻译：对不同教学策略进行随机实验比较，可为教师决策提供有价值的实证依据。然而，传统实验缺乏快速利用数据增加实验学生接触最优条件的清晰路径。借鉴领先科技公司将机器学习和实验方法应用于产品开发的经验，我们探索了自适应实验如何助力课程持续改进。在自适应实验中，当不同实验组/条件被部署给学生时，数据会被实时分析并用于优化后续学生的学习体验。通过机器学习算法识别哪些策略更有利于改善学生体验或学习成效，该算法可动态分配最有效的条件给后续学生，从而更好地满足个性化需求。我们以计算机科学导论课程（CS1）在线作业中自我解释提示的传统实验与自适应实验对比为例，进行案例研究，初步验证了该方法在弥合研究与实践鸿沟、实现持续改进方面的潜力。