Large language models demonstrate remarkable ability in factual recall, yet the fundamental limits of storing and retrieving input--output associations with neural networks remain unclear. We study these limits in a minimal setting: a linear associative memory that maps $p$ input embeddings in $\mathbb{R}^d$ to their corresponding~$d$-dimensional targets via a single layer, requiring each mapped input to be well separated from all other targets. Unlike in supervised classification, this strict separation induces~$p$ constraints per association and produces strong correlations between constraints that make a direct characterisation of the storage capacity difficult. Here, we provide a precise characterisation of this capacity in the following way. We first introduce a decoupled model in which each input has its own independent set of competing outputs, and provide numerical and analytical evidence that this decoupled model is equivalent to the original model in terms of storage capacity, spectra of the learnt weights, and storage mechanism. Using tools from statistical physics, we show that the decoupled model can store up to $p_c \log p_c / d^2 = 1 / 2$ associations, and generalise the computation of $p_c$ to linear two-layer architectures. Our analysis also gives mechanistic insight into how the optimal solution improves over a naïve Hebbian learning rule: rather than boosting input-output alignments with broad fluctuations, the optimal solution raises the correct scores just above the extreme-value threshold set by the competing outputs. These findings give a sharp statistical-physics characterisation of factual storage in linear networks and provide a baseline for understanding the memory capacity of more realistic neural architectures.

翻译：大型语言模型在事实回忆方面展现出卓越能力，但神经网络存储与检索输入-输出关联的根本限制仍不明确。我们在一个最小化设定中研究这些限制：一种线性联想记忆通过单层网络将 $\mathbb{R}^d$ 中的 $p$ 个输入嵌入映射至其对应的 $d$ 维目标，要求每个映射输入与其他所有目标严格分离。与监督分类不同，这种严格分离为每个关联引入 $p$ 个约束，并产生约束间的强相关性，使得直接刻画存储容量变得困难。本文通过以下方式精确刻画该容量：首先引入一个解耦模型，其中每个输入拥有独立的一组竞争输出，数值与理论证据表明该解耦模型在存储容量、学习权重谱及存储机制方面与原模型等价。利用统计物理工具，我们证明解耦模型最多可存储 $p_c \log p_c / d^2 = 1 / 2$ 个关联，并将 $p_c$ 的计算推广至线性双层架构。分析还揭示了最优解比朴素Hebbian学习规则更优的机制：并非通过广泛波动提升输入-输出对齐，而是将正确分数恰好提升至竞争输出设定的极值阈值之上。这些发现为线性网络中的事实存储提供了尖锐的统计物理刻画，并为理解更真实神经架构的记忆容量建立了基准。