Crowdsourcing is a popular method used to estimate ground-truth labels by collecting noisy labels from workers. In this work, we are motivated by crowdsourcing applications where each worker can exhibit two levels of accuracy depending on a task's type. Applying algorithms designed for the traditional Dawid-Skene model to such a scenario results in performance which is limited by the hard tasks. Therefore, we first extend the model to allow worker accuracy to vary depending on a task's unknown type. Then we propose a spectral method to partition tasks by type. After separating tasks by type, any Dawid-Skene algorithm (i.e., any algorithm designed for the Dawid-Skene model) can be applied independently to each type to infer the truth values. We theoretically prove that when crowdsourced data contain tasks with varying levels of difficulty, our algorithm infers the true labels with higher accuracy than any Dawid-Skene algorithm. Experiments show that our method is effective in practical applications.

翻译：众包是一种通过收集工人噪声标签来估计真实标签的流行方法。在本工作中，我们受众包应用场景启发，其中每个工人可根据任务类型表现出两种准确度水平。将传统Dawid-Skene模型设计的算法应用于此类场景时，其性能会受到困难任务的限制。因此，我们首先扩展模型，允许工人准确度根据任务的未知类型而变化。随后提出一种谱方法以按类型划分任务。在按类型分离任务后，任何Dawid-Skene算法（即任何为Dawid-Skene模型设计的算法）均可独立应用于每种类型以推断真值。我们从理论上证明：当众包数据包含难度不同的任务时，我们的算法能比任何Dawid-Skene算法更准确地推断真实标签。实验表明，该方法在实际应用中效果显著。