Predicting future outcomes is a prevalent application of machine learning in social impact domains. Examples range from predicting student success in education to predicting disease risk in healthcare. Practitioners recognize that the ultimate goal is not just to predict but to act effectively. Increasing evidence suggests that relying on outcome predictions for downstream interventions may not have desired results. In most domains there exists a multitude of possible interventions for each individual, making the challenge of taking effective action more acute. Even when causal mechanisms connecting the individual's latent states to outcomes is well understood, in any given instance (a specific student or patient), practitioners still need to infer -- from budgeted measurements of latent states -- which of many possible interventions will be most effective for this individual. With this in mind, we ask: when are accurate predictors of outcomes helpful for identifying the most suitable intervention? Through a simple model encompassing actions, latent states, and measurements, we demonstrate that pure outcome prediction rarely results in the most effective policy for taking actions, even when combined with other measurements. We find that except in cases where there is a single decisive action for improving the outcome, outcome prediction never maximizes "action value", the utility of taking actions. Making measurements of actionable latent states, where specific actions lead to desired outcomes, considerably enhances the action value compared to outcome prediction, and the degree of improvement depends on action costs and the outcome model. This analysis emphasizes the need to go beyond generic outcome prediction in interventional settings by incorporating knowledge of plausible actions and latent states.
翻译:预测未来结果是机器学习在社会影响领域中的一种广泛应用。例如,从教育中预测学生成功到医疗中预测疾病风险。从业者认识到,最终目标不仅是预测,而是有效行动。越来越多的证据表明,依赖结果预测进行下游干预可能不会产生预期效果。在大多数领域中,每个个体都存在多种可能的干预措施,这使得采取有效行动的挑战更加严峻。即使连接个体潜在状态与结果的因果机制被充分理解,在任何特定实例(如某个具体学生或患者)中,从业者仍需从预算有限的潜在状态测量中推断出多种可能的干预措施中哪一个对该个体最有效。基于此,我们提出疑问:准确的结果预测在何种情况下有助于识别最合适的干预措施?通过一个包含行动、潜在状态和测量的简单模型,我们证明纯结果预测很少能产生最有效的行动策略,即使结合其他测量也是如此。我们发现,除非存在一个决定性的单一行动来改善结果,否则结果预测从未最大化“行动价值”(即采取行动的效用)。对可行动的潜在状态进行测量(其中特定行动可导向期望结果)能显著提升行动价值,优于结果预测,且提升程度取决于行动成本与结果模型。这一分析强调,在干预环境中需超越通用结果预测,融入对可行行动和潜在状态的知识。