Adding memory to pretrained language models typically requires architectural changes or weight modification. We present Prometheus Mind, which retrofits memory to a frozen Qwen3-4B using 11 modular adapters (530MB, 7% overhead) -- fully reversible by removing the adapters. Building this system required solving four problems: (1) Extraction -- we develop Contrastive Direction Discovery (CDD), which finds semantic directions via minimal pairs without labeled data. (2) Training -- end-to-end optimization collapses; stage-wise training of each adapter on simple proxy tasks succeeds. (3) Injection -- learned encoders fail to generalize; we find that lm_head-weight rows already provide the mapping we need, requiring no training. (4) Hidden state collapse -- transformers make ``wife'' and ``brother'' 0.98+ similar; we train projections to recover distinction (0.98 $\rightarrow$ 0.09). On PrometheusExtract-132 (132 cases), the system achieves 94.4% retrieval on clean inputs (n=54, 95% CI: [84.9%, 98.1%]), degrading to 19.4% on informal inputs with ellipsis, filler words, or implicit subjects (n=36). The primary bottleneck is relation classification (47.3% accuracy), responsible for most extraction errors.
翻译:为预训练语言模型添加记忆功能通常需要架构修改或权重调整。我们提出普罗米修斯之心系统,通过11个模块化适配器(530MB,7%开销)为冻结的Qwen3-4B模型添加记忆功能——移除适配器即可完全复原。构建该系统需要解决四个关键问题:(1)提取——我们开发对比方向发现方法,通过最小配对在无标注数据条件下发现语义方向。(2)训练——端到端优化会崩溃;通过简单代理任务分阶段训练每个适配器获得成功。(3)注入——学习到的编码器泛化能力不足;我们发现lm_head权重行已提供所需映射,无需额外训练。(4)隐藏状态坍缩——Transformer使"妻子"与"兄弟"的相似度达0.98+;我们训练投影矩阵以恢复区分度(0.98 $\rightarrow$ 0.09)。在PrometheusExtract-132测试集(132个案例)上,系统在规范输入中实现94.4%检索率(n=54,95%置信区间:[84.9%,98.1%]),在包含省略、填充词或隐含主语的非规范输入中降至19.4%(n=36)。主要瓶颈在于关系分类任务(47.3%准确率),该环节导致了大多数提取错误。