Crowd-sourcing deals with solving problems by assigning them to a large number of non-experts called crowd using their spare time. In these systems, the final answer to the question is determined by summing up the votes obtained from the community. The popularity of using these systems has increased by facilitation of access to community members through mobile phones and the Internet. One of the issues raised in crowd-sourcing is how to choose people and how to collect answers. Usually, the separation of users is done based on their performance in a pre-test. Designing the pre-test for performance calculation is challenging; The pre-test questions should be chosen in a way that they test the characteristics in people related to the main questions. One of the ways to increase the accuracy of crowd-sourcing systems is to pay attention to people's cognitive characteristics and decision-making model to form a crowd and improve the estimation of the accuracy of their answers to questions. People can estimate the correctness of their responses while making a decision. The accuracy of this estimate is determined by a quantity called metacognition ability. Metacoginition is referred to the case where the confidence level is considered along with the answer to increase the accuracy of the solution. In this paper, by both mathematical and experimental analysis, we would answer the following question: Is it possible to improve the performance of the crowd-sourcing system by knowing the metacognition of individuals and recording and using the users' confidence in their answers?
翻译:众包通过将任务分配给大量利用闲暇时间的非专家群体(即众包者)来解决问题。在这些系统中,问题的最终答案通过汇总社区获得的投票来确定。通过手机和互联网便捷接入社区成员的方式,提升了此类系统的普及度。众包中的一个关键问题是如何选择参与者以及如何收集答案。通常,用户的分选基于其在预测试中的表现。设计用于计算表现的预测试具有挑战性:预测试问题的选择应能检验参与者与主要问题相关的特质。提升众包系统准确性的方法之一是关注个体的认知特征与决策模型,以优化群体构成并改进对其问题回答准确率的估计。个体在决策时能够评估自身回答的正确性,该评估的准确性由称为元认知能力的量值决定。元认知指在提供答案的同时考虑置信度水平以提高解决方案的准确性。本文通过数学分析与实验分析,旨在回答以下问题:通过了解个体的元认知能力并记录使用其答案置信度,是否能够提升众包系统的性能?