Foundation models have transformed machine learning through large-scale pretraining and increased test-time compute. Despite surpassing human performance in several domains, these models remain fundamentally limited in continuous operation, experience accumulation, and personalization, capabilities that are central to adaptive intelligence. While continual learning research has long targeted these goals, its historical focus on in-weight learning (IWL), i.e., updating a single model's parameters to absorb new knowledge, has rendered catastrophic forgetting a persistent challenge. Our position is that combining the strengths of In-Weight Learning (IWL) and the newly emerged capabilities of In-Context Learning (ICL) through the design of modular memory is the missing piece for continual adaptation at scale. We outline a conceptual framework for modular memory-centric architectures that leverage ICL for rapid adaptation and knowledge accumulation, and IWL for stable updates to model capabilities, charting a practical roadmap toward continually learning agents.

翻译：摘要：基础模型通过大规模预训练和增加测试时计算能力，已深刻改变了机器学习领域。尽管在多个领域超越了人类表现，但这类模型在持续运行、经验积累与个性化适应等自适应智能核心能力上仍存在根本性局限。持续学习研究虽长期致力于实现这些目标，但其历史上对权重内学习（IWL）的侧重（即通过更新单一模型参数来吸收新知识），使得灾难性遗忘成为持久性挑战。本文的立场是：通过模块化记忆设计，将权重内学习（IWL）的优势与新近涌现的上下文内学习（ICL）能力相结合，正是实现大规模持续适应的关键要素。我们提出一个以模块化记忆为中心的架构概念框架，该框架利用ICL实现快速适应与知识积累，并借助IWL进行模型能力的稳定更新，为构建持续学习智能体绘制了实用路线图。