Large Language Models (LLMs) generalize across tasks via reusable representations and flexible reasoning, yet remain brittle in real deployment under evolving tasks and continual distribution shift. A common approach is Test-Time Adaptation (TTA), existing ones of which updates models with hand-designed unsupervised objectives over the full parameter space and mostly overlook preserving shared source knowledge and the reliability of adaptation signals. Drawing on molecular signaling cascades of memory updating in Drosophila, we propose Synapse Consolidation (SyCo), a parameter-efficient LLM adaptation method that updates low-rank adapters through Rac1 and MAPK pathways under the guidance of a structured TTA objective driven by problem understanding, process understanding, and source-domain guardrail. Rac1 confines plasticity to a tail-gradient subspace that is less critical for source knowledge, enabling rapid specialization while preserving source representations. MAPK uses a tiered controller to suppress noisy updates and consolidate useful adaptations under non-stationary streams. To model real deployments with multiple sources and continually emerging tasks, we introduce Multi-source Open-set Adaptation (MOA) setting, where a model is trained on multiple labeled source tasks and then adapts on open, non-stationary unlabeled test streams that mix seen and unseen tasks with partial overlap in label and intent space. Across 18 NLP datasets and the MOA setting, SyCo consistently outperforms strong baselines, achieving 78.31\% on unseen-task adaptation and 85.37\% on unseen-data shifts.
翻译:大型语言模型(LLMs)通过可复用表征和灵活推理实现跨任务泛化,但在动态任务与持续分布偏移的真实部署场景中仍显脆弱。现有测试时自适应方法通常采用基于手工设计的无监督目标在全参数空间更新模型,且大多忽视保留共享源知识及自适应信号的可靠性。受果蝇记忆更新的分子信号级联机制启发,我们提出突触巩固(SyCo)——一种参数高效的LLM自适应方法,通过Rac1和MAPK通路更新低秩适配器,并在由问题理解、过程理解及源域防护栏构成的结构化TTA目标引导下运行。Rac1将可塑性限制在对源知识次要的尾梯度子空间,在保留源表征的同时实现快速特化;MAPK采用分层控制器抑制噪声更新,并在非平稳流中巩固有效自适应。为模拟包含多源任务与持续涌现任务的真实部署场景,我们引入多源开放集自适应(MOA)设定:模型先在多个带标签源任务上训练,随后在混合已知/未知任务且标签与意图空间部分重叠的开放非平稳无标签测试流中进行自适应。在18个NLP数据集及MOA设定下,SyCo持续超越强基线方法,在未知任务自适应与未知数据迁移场景中分别取得78.31%和85.37%的性能表现。