Prediction algorithms that quantify the expected benefit of a given treatment conditional on patient characteristics can critically inform medical decisions. Quantifying the performance of treatment benefit prediction algorithms is an active area of research. A recently proposed metric, the concordance statistic for benefit (cfb), evaluates the discriminative ability of a treatment benefit predictor by directly extending the concept of the concordance statistic from a risk model with a binary outcome to a model for treatment benefit. In this work, we scrutinize $cfb$ on multiple fronts. Through numerical examples and theoretical developments, we show that cfb is not a proper scoring rule. We also show that it is sensitive to the unestimable correlation between counterfactual outcomes and to the definition of matched pairs. We argue that measures of statistical dispersion applied to predicted benefits do not suffer from these issues and can be an alternative metric for the discriminatory performance of treatment benefit predictors.

翻译：量化给定治疗基于患者特征的预期获益的预测算法，可对医疗决策提供关键信息。量化治疗获益预测算法的性能是一个活跃的研究领域。近期提出的度量指标——获益一致性统计量(cfb)——通过将一致性统计量的概念从二元结局的风险模型直接扩展至治疗获益模型，来评估治疗获益预测变量的鉴别能力。本研究从多个角度对$cfb$进行审视。通过数值示例和理论推导，我们表明cfb并非恰当的评分规则。同时证明其对反事实结果之间不可估的相关性以及匹配对定义敏感。我们认为，应用于预测获益的统计离散度量不存在这些问题，并可作为治疗获益预测变量鉴别能力的替代指标。