Assessing forecasting proficiency is a time-intensive activity, often requiring us to wait months or years before we know whether or not the reported forecasts were good. In this study, we develop adaptive cognitive tests that predict forecasting proficiency without the need to wait for forecast outcomes. Our procedures provide information about which cognitive tests to administer to each individual, as well as how many cognitive tests to administer. Using item response models, we identify and tailor cognitive tests to assess forecasters of different skill levels, aiming to optimize accuracy and efficiency. We show how the procedures can select highly-informative cognitive tests from a larger battery of tests, reducing the time taken to administer the tests. We use a second, independent dataset to show that the selected tests yield scores that are highly related to forecasting proficiency. This approach enables real-time, adaptive testing, providing immediate insights into forecasting talent in practical contexts.

翻译：评估预测能力是一项耗时活动，通常需要等待数月或数年才能确定所报告的预测是否准确。本研究开发了自适应认知测试，无需等待预测结果即可评估预测能力。我们的方法提供了针对每个个体应实施哪些认知测试以及实施多少认知测试的信息。通过项目反应模型，我们识别并定制认知测试以评估不同技能水平的预测者，旨在优化准确性和效率。我们展示了该方法如何从大量测试中选择信息量高的认知测试，从而减少测试实施时间。利用第二个独立数据集，我们证明所选测试产生的分数与预测能力高度相关。该方法支持实时自适应测试，为实际场景中的预测人才评估提供即时洞察。