Large language models (LLMs) are increasingly used in statistical research and applications. However,they are also notorious for unreliable or biased information. Here, we explore whether LLMs can be used to improve the precision of randomized controlled trials (RCTs) in a safe and rigorous way. Following similar work on leveraging observational data, we incorporate LLM predictions into an RCT analysis. While incorporating external predictions to improve precision is not new, the value of using LLM predictions in this manner is an open question. We develop a pipeline for best leveraging LLM predictions in this context and apply it to three different case studies. We find that these predictions can safely improve precision, particularly when the RCT lacks predictive covariates or contains covariates, such as text data, that are well-suited to LLMs.

翻译：大型语言模型（LLMs）在统计研究和应用中日益广泛，但也因其不稳定或存在偏差的信息而闻名。本文探讨了能否以安全且严谨的方式利用LLMs提升随机对照试验（RCTs）的精确度。借鉴类似利用观测数据的研究工作，我们将LLM预测结果纳入RCT分析。尽管整合外部预测以提升精确度并非新概念，但以这种方式使用LLM预测的价值仍是一个开放性问题。我们开发了一套在该背景下最优利用LLM预测的流程，并将其应用于三个不同的案例研究。结果表明，这些预测能安全提升精确度，尤其在RCT缺乏预测性协变量或包含文本数据等适合LLM处理的协变量时效果显著。