Embedding models group text by semantic content, what text is about. We show that temporal co-occurrence within texts discovers a different kind of structure: recurrent transition-structure concepts or what text does. We train a 29.4M-parameter contrastive model on 373 million co-occurrence pairs from 9,766 Project Gutenberg texts (24.96 million passages), mapping pre-trained embeddings into an association space where passages with similar transition structure cluster together. Under capacity constraint (42.75% accuracy), the model must compress across recurring patterns rather than memorise individual co-occurrences. Clustering at six granularities (k=50 to k=2,000) produces a multi-resolution concept map; from broad modes like "direct confrontation" and "lyrical meditation" to precise registers and scene templates like "sailor dialect" and "courtroom cross-examination." At k=100, clusters average 4,508 books each (of 9,766), confirming corpus-wide patterns. Direct comparison with embedding-similarity clustering shows that raw embeddings group by topic while association-space clusters group by function, register, and literary tradition. Unseen novels are assigned to existing clusters without retraining; the association model concentrates each novel into a selective subset of coherent clusters, while raw embedding assignment saturates nearly all clusters. Validation controls address positional, length, and book-concentration confounds. The method extends Predictive Associative Memory (PAM, arXiv:2602.11322) from episodic recall to concept formation: where PAM recalls specific associations, multi-epoch contrastive training under compression extracts structural patterns that transfer to unseen texts, the same framework producing qualitatively different behaviour in a different regime.
翻译:嵌入模型根据文本的语义内容(即文本"关于什么")对文本进行分组。我们证明,文本内的时间共现能够发现一种不同类型的结构:循环出现的过渡结构概念,即文本"做什么"。我们在来自9,766个古腾堡计划文本(2,496万个段落)的3.73亿个共现对上训练了一个29.4百万参数的对比模型,将预训练的嵌入映射到一个关联空间,其中具有相似过渡结构的段落聚集在一起。在容量约束下(42.75%准确率),模型必须压缩跨重复出现的模式,而非记忆单个共现。在六种粒度(k=50至k=2,000)上进行聚类,生成一个多分辨率概念图:从"直接对抗"和"抒情沉思"等广义模式,到"水手方言"和"法庭交叉质询"等精确语域和场景模板。在k=100时,每个聚类平均涵盖4,508本书(共9,766本),证实了语料库范围内的模式。与嵌入相似性聚类的直接比较表明,原始嵌入按主题分组,而关联空间中的聚类则按功能、语域和文学传统分组。新小说无需重新训练即可分配到现有聚类;关联模型将每部小说集中到一组选定的连贯聚类中,而原始嵌入分配则几乎饱和所有聚类。验证控制解决了位置、长度和书籍集中度的混淆因素。该方法将预测性联想记忆(PAM, arXiv:2602.11322)从情节回忆扩展到概念形成:PAM回忆特定关联,而压缩下的多周期对比训练提取出可迁移到未见文本的结构模式——同一框架在不同机制下产生性质不同的行为。