Quantitative risk assessments have been used for decades to help inform criminal justice decisions. A set of predictors and a categorical response variable are used to train a statistical classifier. The trained classifier can be employed to compute risk scores for a new, unlabeled case for which a forecast is needed, one risk score for each response variable class. In the spirit of Bayes classifiers, the class with the largest risk score conventionally becomes the forecasted outcome label for that case. The largest and next largest risk scores can be very similar or very different. However, when they are much the same, the outcome class is essentially being assigned by a coin flip; the classification is unreliable. Forecasting error becomes more likely, and concerns about fairness can arise. This paper shows how Mondrian conformal prediction sets can increase classification reliability when the classifier is indecisive and provide valid estimates of forecast uncertainty. Risk forecasts for a large dataset of offenders on probation are used to illustrate the issues.

翻译：暂无翻译