Autonomous AI agents operating in dynamic environments face a persistent challenge: acquiring new capabilities without erasing prior knowledge. We present Adaptive Memory Crystallization (AMC), a memory architecture for progressive experience consolidation in continual reinforcement learning. AMC is conceptually inspired by the qualitative structure of synaptic tagging and capture (STC) theory, the idea that memories transition through discrete stability phases, but makes no claim to model the underlying molecular or synaptic mechanisms. AMC models memory as a continuous crystallization process in which experiences migrate from plastic to stable states according to a multi-objective utility signal. The framework introduces a three-phase memory hierarchy (Liquid--Glass--Crystal) governed by an Itô stochastic differential equation (SDE) whose population-level behavior is captured by an explicit Fokker--Planck equation admitting a closed-form Beta stationary distribution. We provide proofs of: (i) well-posedness and global convergence of the crystallization SDE to a unique Beta stationary distribution; (ii) exponential convergence of individual crystallization states to their fixed points, with explicit rates and variance bounds; and (iii) end-to-end Q-learning error bounds and matching memory-capacity lower bounds that link SDE parameters directly to agent performance. Empirical evaluation on Meta-World MT50, Atari 20-game sequential learning, and MuJoCo continual locomotion consistently shows improvements in forward transfer (+34--43\% over the strongest baseline), reductions in catastrophic forgetting (67--80\%), and a 62\% decrease in memory footprint.
翻译:在动态环境中运行的自主AI智能体面临一个持久性挑战:在习得新能力的同时不遗忘已有知识。我们提出自适应记忆结晶(AMC),一种用于持续强化学习中经验渐进式固化的记忆架构。AMC的概念灵感源自突触标记与捕获(STC)理论的定性结构——该理论认为记忆通过离散的稳定性阶段进行转换,但本文并未声称对底层分子或突触机制进行建模。AMC将记忆建模为连续结晶过程,其中经验根据多目标效用信号从可塑状态迁移至稳定状态。该框架引入由伊藤随机微分方程(SDE)控制的三阶段记忆层次(液态-玻璃态-晶态),其总体行为由具有封闭形式贝塔平稳分布的显式福克-普朗克方程描述。我们提供以下证明:(i)结晶SDE适定性与全局收敛至唯一贝塔平稳分布;(ii)各结晶状态以显式速率与方差界指数收敛至不动点;(iii)端到端Q学习误差界与匹配记忆容量下界,直接建立SDE参数与智能体性能的关联。在Meta-World MT50、Atari 20游戏序列学习及MuJoCo持续运动任务上的实证评估一致显示:前向迁移提升34-43%(超过最强基线),灾难性遗忘减少67-80%,且内存占用降低62%。