Artificial intelligence systems, particularly large language models (LLMs), are increasingly being employed in high-stakes decisions that impact both individuals and society at large, often without adequate safeguards to ensure safety, quality, and equity. Yet LLMs hallucinate, lack common sense, and are biased - shortcomings that may reflect LLMs' inherent limitations and thus may not be remedied by more sophisticated architectures, more data, or more human feedback. Relying solely on LLMs for complex, high-stakes decisions is therefore problematic. Here we present a hybrid collective intelligence system that mitigates these risks by leveraging the complementary strengths of human experience and the vast information processed by LLMs. We apply our method to open-ended medical diagnostics, combining 40,762 differential diagnoses made by physicians with the diagnoses of five state-of-the art LLMs across 2,133 medical cases. We show that hybrid collectives of physicians and LLMs outperform both single physicians and physician collectives, as well as single LLMs and LLM ensembles. This result holds across a range of medical specialties and professional experience, and can be attributed to humans' and LLMs' complementary contributions that lead to different kinds of errors. Our approach highlights the potential for collective human and machine intelligence to improve accuracy in complex, open-ended domains like medical diagnostics.

翻译：人工智能系统，特别是大型语言模型（LLM），正越来越多地应用于影响个人和整个社会的高风险决策中，且往往缺乏足够的安全保障来确保其安全性、质量和公平性。然而，LLM会产生幻觉、缺乏常识且存在偏见——这些缺陷可能反映了LLM固有的局限性，因此可能无法通过更复杂的架构、更多数据或更多人类反馈来弥补。因此，完全依赖LLM进行复杂的高风险决策是有问题的。在此，我们提出一种混合集体智能系统，通过利用人类经验的互补优势和LLM处理的海量信息来缓解这些风险。我们将该方法应用于开放式医学诊断，将医生做出的40,762项鉴别诊断与五种最先进LLM在2,133个医疗案例中的诊断相结合。研究表明，医生与LLM组成的混合集体在诊断准确性上优于单个医生、医生集体、单个LLM以及LLM集合。这一结果在不同医学专业和职业经验范围内均成立，可归因于人类和LLM的互补贡献导致了不同类型的错误。我们的方法凸显了人机集体智能在医学诊断等复杂开放领域提高准确性的潜力。