Machine learning (ML) models have been quite successful in predicting outcomes in many applications. However, in some cases, domain experts might have a judgment about the expected outcome that might conflict with the prediction of ML models. One main reason for this is that the training data might not be totally representative of the population. In this paper, we present a novel framework that aims at leveraging experts' judgment to mitigate the conflict. The underlying idea behind our framework is that we first determine, using a generative adversarial network, the degree of representation of an unlabeled data point in the training data. Then, based on such degree, we correct the \textcolor{black}{machine learning} model's prediction by incorporating the experts' judgment into it, where the higher that aforementioned degree of representation, the less the weight we put on the expert intuition that we add to our corrected output, and vice-versa. We perform multiple numerical experiments on synthetic data as well as two real-world case studies (one from the IT services industry and the other from the financial industry). All results show the effectiveness of our framework; it yields much higher closeness to the experts' judgment with minimal sacrifice in the prediction accuracy, when compared to multiple baseline methods. We also develop a new evaluation metric that combines prediction accuracy with the closeness to experts' judgment. Our framework yields statistically significant results when evaluated on that metric.

翻译：机器学习（ML）模型在众多应用场景中成功预测结果。然而在某些情况下，领域专家对预期结果的判断可能与ML模型预测产生冲突。其主要原因之一是训练数据可能无法完全代表总体分布。本文提出一种新型框架，旨在利用专家判断缓解此类冲突。该框架的核心思想是：首先通过生成对抗网络确定未标记数据点在训练数据中的代表性程度，然后基于该程度通过融入专家判断修正ML模型预测结果——当上述代表性程度越高时，我们在修正输出中赋予专家直觉的权重越低，反之亦然。我们在合成数据及两个真实案例研究（分别来自IT服务业和金融业）中进行了多项数值实验。所有结果均验证了该框架的有效性：与多种基准方法相比，该方法在最小牺牲预测精度的前提下，实现了与专家判断更高程度的契合。我们还开发了结合预测精度与专家判断契合度的新型评估指标。在该指标评估下，我们的框架取得了统计显著的效果。