Harnessing collective intelligence to drive effective decision-making and collaboration benefits from the ability to detect and characterize heterogeneity in consensus beliefs. This is particularly true in domains such as technology acceptance or leadership perception, where a consensus defines an intersubjective truth, leading to the possibility of multiple "ground truths" when subsets of respondents sustain mutually incompatible consensuses. Cultural Consensus Theory (CCT) provides a statistical framework for detecting and characterizing these divergent consensus beliefs. However, it is unworkable in modern applications because it lacks the ability to generalize across even highly similar beliefs, is ineffective with sparse data, and can leverage neither external knowledge bases nor learned machine representations. Here, we overcome these limitations through Infinite Deep Latent Construct Cultural Consensus Theory (iDLC-CCT), a nonparametric Bayesian model that extends CCT with a latent construct that maps between pretrained deep neural network embeddings of entities and the consensus beliefs regarding those entities among one or more subsets of respondents. We validate the method across domains including perceptions of risk sources, food healthiness, leadership, first impressions, and humor. We find that iDLC-CCT better predicts the degree of consensus, generalizes well to out-of-sample entities, and is effective even with sparse data. To improve scalability, we introduce an efficient hard-clustering variant of the iDLC-CCT using an algorithm derived from a small-variance asymptotic analysis of the model. The iDLC-CCT, therefore, provides a workable computational foundation for harnessing collective intelligence under a lack of cultural consensus and may potentially form the basis of consensus-aware information technologies.

翻译：利用集体智慧推动有效决策和协作，依赖于检测和表征共识信念异质性的能力。这一点在技术接受度或领导力感知等领域尤为突出——在这些领域中，共识定义了主体间真理，当受访者子集持有互不相容的共识时，便可能出现多个“基本事实”。文化共识理论（CCT）为检测和表征这些分歧的共识信念提供了统计框架。然而，该理论在现代应用中难以奏效，因为它缺乏跨高度相似信念的泛化能力，对稀疏数据无效，且无法利用外部知识库或习得的机器表征。在此，我们通过无限深度潜结构文化共识理论（iDLC-CCT）克服了这些局限：这是一种非参数贝叶斯模型，通过引入潜结构，将预训练的深度神经网络实体嵌入与一个或多个受访者子集对这些实体的共识信念进行映射。我们在风险源感知、食品健康度、领导力、第一印象和幽默感等多个领域验证了该方法。研究发现，iDLC-CCT能更准确地预测共识程度，对样本外实体具有良好的泛化能力，且在稀疏数据下依然有效。为提升可扩展性，我们基于模型的小方差渐近分析推导出算法，提出了iDLC-CCT的高效硬聚类变体。因此，iDLC-CCT为在缺乏文化共识下利用集体智慧提供了可行的计算基础，并可能构成共识感知型信息技术的基石。