Large language models (LLMs) such as GPT-3 and ChatGPT have recently demonstrated impressive results across a wide range of tasks. LLMs are still limited, however, in that they frequently fail at complex reasoning, their reasoning processes are opaque, they are prone to 'hallucinate' facts, and there are concerns about their underlying biases. Letting models verbalize reasoning steps as natural language, a technique known as chain-of-thought prompting, has recently been proposed as a way to address some of these issues. Here we present the first release of ThoughtSource, a meta-dataset and software library for chain-of-thought (CoT) reasoning. The goal of ThoughtSource is to improve future artificial intelligence systems by facilitating qualitative understanding of CoTs, enabling empirical evaluations, and providing training data. This first release of ThoughtSource integrates six scientific/medical, three general-domain and five math word question answering datasets.

翻译：大型语言模型（如GPT-3和ChatGPT）近期在广泛任务中展现出显著成果。然而，这些模型仍存在局限：它们在复杂推理中频繁出错、推理过程不透明、易产生事实“幻觉”，且其潜在偏见引发担忧。让模型将推理步骤以自然语言形式进行表述（即思维链提示技术），近期被提出作为应对上述问题的方法之一。本文呈现ThoughtSource的首个版本——一个用于思维链推理的元数据集与软件库。ThoughtSource旨在通过促进对思维链的定性理解、支持经验性评估以及提供训练数据，来改进未来的人工智能系统。该首个版本整合了六个科学/医学领域数据集、三个通用领域数据集和五个数学文字问答数据集。