Clinical prediction models are statistical or machine learning models used to quantify the risk of a certain health outcome using patient data. These can then inform potential interventions on patients, causing an effect called performative prediction: predictions inform interventions which influence the outcome they were trying to predict, leading to a potential underestimation of risk in some patients if a model is updated on this data. One suggested resolution to this is the use of hold-out sets, in which a set of patients do not receive model derived risk scores, such that a model can be safely retrained. We present an overview of clinical and research ethics regarding potential implementation of hold-out sets for clinical prediction models in health settings. We focus on the ethical principles of beneficence, non-maleficence, autonomy and justice. We also discuss informed consent, clinical equipoise, and truth-telling. We present illustrative cases of potential hold-out set implementations and discuss statistical issues arising from different hold-out set sampling methods. We also discuss differences between hold-out sets and randomised control trials, in terms of ethics and statistical issues. Finally, we give practical recommendations for researchers interested in the use hold-out sets for clinical prediction models.

翻译：临床预测模型是用于通过患者数据量化特定健康结局风险的统计或机器学习模型。这些模型可指导对患者的潜在干预，从而引发一种称为"表现性预测"的效应：预测指导干预，而干预反过来影响模型试图预测的结局，导致若基于此类数据更新模型，部分患者的风险可能被低估。对此，一种建议解决方案是使用保留集，即一组患者不接收模型衍生的风险评分，以便模型可安全地重新训练。我们概述了在医疗环境中实施临床预测模型保留集所涉及的临床与研究伦理问题。重点关注善行、无害、自主和公正四项伦理原则，并讨论知情同意、临床均势及诚实告知。通过潜在保留集实施的示例性案例，分析不同保留集抽样方法引发的统计学问题。进一步从伦理与统计学角度讨论保留集与随机对照试验的差异。最后，为有意在临床预测模型中使用保留集的研究者提供实践建议。