Machine learning (ML) models have been quite successful in predicting outcomes in many applications. However, in some cases, domain experts might have a judgment about the expected outcome that might conflict with the prediction of ML models. One main reason for this is that the training data might not be totally representative of the population. In this paper, we present a novel framework that aims at leveraging experts' judgment to mitigate the conflict. The underlying idea behind our framework is that we first determine, using a generative adversarial network, the degree of representation of an unlabeled data point in the training data. Then, based on such degree, we correct the \textcolor{black}{machine learning} model's prediction by incorporating the experts' judgment into it, where the higher that aforementioned degree of representation, the less the weight we put on the expert intuition that we add to our corrected output, and vice-versa. We perform multiple numerical experiments on synthetic data as well as two real-world case studies (one from the IT services industry and the other from the financial industry). All results show the effectiveness of our framework; it yields much higher closeness to the experts' judgment with minimal sacrifice in the prediction accuracy, when compared to multiple baseline methods. We also develop a new evaluation metric that combines prediction accuracy with the closeness to experts' judgment. Our framework yields statistically significant results when evaluated on that metric.

翻译：机器学习（ML）模型在众多应用中预测结果方面取得了显著成功。然而，在某些情况下，领域专家可能对预期结果持有与ML模型预测相冲突的判断。其主要原因之一是训练数据可能无法完全代表总体分布。本文提出了一种新框架，旨在利用专家判断来缓解这一冲突。该框架的核心思想是：首先，使用生成对抗网络确定未标记数据点在训练数据中的代表性程度；然后，基于该程度，通过融入专家判断来修正机器学习模型的预测——代表性程度越高，修正输出时赋予专家直觉的权重越低，反之亦然。我们在合成数据以及两个真实案例（分别来自IT服务行业和金融行业）上进行了多项数值实验。所有结果均显示该框架的有效性：与多种基线方法相比，该方法在极小牺牲预测精度的前提下，显著提升了与专家判断的一致性。我们还开发了一个结合预测精度与专家判断一致性的新评估指标。在该指标上，我们的框架取得了统计显著的结果。