How do large language models (LLMs) know what they know? Answering this question has been difficult because pre-training data is often a "black box" - unknown or inaccessible. The recent release of nanochat - a family of small LLMs with fully open pre-training data - addresses this as it provides a transparent view into where a model's parametric knowledge comes from. Towards the goal of understanding how knowledge is encoded by LLMs, we release NanoKnow, a benchmark dataset that partitions questions from Natural Questions and SQuAD into splits based on whether their answers are present in nanochat's pre-training corpus. Using these splits, we can now properly disentangle the sources of knowledge that LLMs rely on when producing an output. To demonstrate NanoKnow's utility, we conduct experiments using eight nanochat checkpoints. Our findings show: (1) closed-book accuracy is strongly influenced by answer frequency in the pre-training data, (2) providing external evidence can mitigate this frequency dependence, (3) even with external evidence, models are more accurate when answers were seen during pre-training, demonstrating that parametric and external knowledge are complementary, and (4) non-relevant information is harmful, with accuracy decreasing based on both the position and the number of non-relevant contexts. We release all NanoKnow artifacts at https://github.com/castorini/NanoKnow.

翻译：大型语言模型（LLMs）如何知道它们所知道的内容？回答这个问题一直很困难，因为预训练数据往往是一个“黑箱”——未知或不可访问。最近发布的nanochat——一系列具有完全开放预训练数据的小型LLM——解决了这一问题，因为它提供了模型参数知识来源的透明视图。为了理解LLM如何编码知识，我们发布了NanoKnow，这是一个基准数据集，它将来自Natural Questions和SQuAD的问题根据其答案是否存在于nanochat的预训练语料库中划分为不同的子集。利用这些子集，我们现在可以正确地区分LLM在生成输出时所依赖的知识来源。为了展示NanoKnow的实用性，我们使用八个nanochat检查点进行了实验。我们的发现表明：（1）闭卷准确性受预训练数据中答案频率的强烈影响；（2）提供外部证据可以缓解这种频率依赖性；（3）即使有外部证据，当答案在预训练期间被看到时，模型更准确，这表明参数知识和外部知识是互补的；（4）非相关信息是有害的，准确性随着非相关上下文的位置和数量增加而下降。我们在https://github.com/castorini/NanoKnow发布所有NanoKnow材料。