Weighted Majority Voting (WMV) is a well-known optimal decision rule for collective decision making, given the probability of sources to provide accurate information (trustworthiness). However, in reality, the trustworthiness is not a known quantity to the decision maker - they have to rely on an estimate called trust. A (machine learning) algorithm that computes trust is called unbiased when it has the property that it does not systematically overestimate or underestimate the trustworthiness. To formally analyse the uncertainty to the decision process, we introduce and analyse two important properties of such unbiased trust values: stability of correctness and stability of optimality. Stability of correctness means that the decision accuracy that the decision maker believes they achieved is equal to the actual accuracy. We prove stability of correctness holds. Stability of optimality means that the decisions made based on trust, are equally good as they would have been if they were based on trustworthiness. Stability of optimality does not hold. We analyse the difference between the two, and bounds thereon. We also present an overview of how sensitive decision correctness is to changes in trust and trustworthiness.
翻译:加权多数投票(WMV)是一种在给定信息源提供准确信息的概率(可信度)条件下,集体决策中已知的最优决策规则。然而,在实际应用中,决策者通常无法获知可信度的真实值——他们必须依赖一种称为信任度的估计量。计算信任度的(机器学习)算法若满足不系统性地高估或低估可信度的特性,则被称为无偏估计算法。为形式化分析决策过程中的不确定性,我们引入并分析了此类无偏信任度值的两个重要性质:正确性稳定性与最优性稳定性。正确性稳定性指决策者认为自身达到的决策准确度与实际准确度相等。我们证明了正确性稳定性成立。最优性稳定性指基于信任度作出的决策与基于可信度作出的决策具有同等质量。最优性稳定性并不成立。我们分析了两者间的差异及其界限,并概述了决策正确性对信任度与可信度变化的敏感程度。