Human forecasting accuracy in practice relies on the 'wisdom of the crowd' effect, in which predictions about future events are significantly improved by aggregating across a crowd of individual forecasters. Past work on the forecasting ability of large language models (LLMs) suggests that frontier LLMs, as individual forecasters, underperform compared to the gold standard of a human crowd forecasting tournament aggregate. In Study 1, we expand this research by using an LLM ensemble approach consisting of a crowd of twelve LLMs. We compare the aggregated LLM predictions on 31 binary questions to that of a crowd of 925 human forecasters from a three-month forecasting tournament. Our preregistered main analysis shows that the LLM crowd outperforms a simple no-information benchmark and is not statistically different from the human crowd. In exploratory analyses, we find that these two approaches are equivalent with respect to medium-effect-size equivalence bounds. We also observe an acquiescence effect, with mean model predictions being significantly above 50%, despite an almost even split of positive and negative resolutions. Moreover, in Study 2, we test whether LLM predictions (of GPT-4 and Claude 2) can be improved by drawing on human cognitive output. We find that both models' forecasting accuracy benefits from exposure to the median human prediction as information, improving accuracy by between 17% and 28%: though this leads to less accurate predictions than simply averaging human and machine forecasts. Our results suggest that LLMs can achieve forecasting accuracy rivaling that of human crowd forecasting tournaments: via the simple, practically applicable method of forecast aggregation. This replicates the 'wisdom of the crowd' effect for LLMs, and opens up their use for a variety of applications throughout society.

翻译：人类预测准确性在实践中有赖于“群体智慧”效应——即通过整合一群独立预测者的判断，能显著提升对未来事件的预测表现。以往关于大语言模型（LLM）预测能力的研究表明，作为独立预测者的前沿LLM，其表现不及人类群体预测竞赛的黄金标准（聚合预测结果）。在研究1中，我们通过采用包含十二个LLM的集合方法拓展该研究，将LLM对31个二元问题的聚合预测结果，与来自三个月期预测竞赛中925名人类预测者构成的群体进行对比。我们预先注册的主要分析显示：LLM群体表现优于简单无信息基准，且在统计上未与人类群体存在显著差异。探索性分析发现，两种方法在中等效应量等价界值范围内具有等价性。我们还观察到默许效应——尽管正向与负向结局几乎对半分布，但模型平均预测值显著高于50%。此外，在研究2中，我们检验了LLM预测（GPT-4与Claude 2）能否通过借鉴人类认知输出得到改进。结果表明，当向模型提供人类预测中位数作为信息输入时，两种模型的预测准确性均提升17%至28%，但其准确度仍低于直接平均人类与机器预测的简单方法。本研究结果表明：通过简单且具实践意义的预测聚合方法，LLM能够达到媲美人类群体预测竞赛的准确性。这既在LLM中复现了“群体智慧”效应，也为该技术在社会各领域的应用开辟了新路径。