Continual learning (CL) has been a critical topic in contemporary deep neural network applications, where higher levels of both forward and backward transfer are desirable for an effective CL performance. Existing CL strategies primarily focus on task models, either by regularizing model updates or by separating task-specific and shared components, while often overlooking the potential of leveraging inter-task relationships to enhance transfer. To address this gap, we propose a transferability-aware task embedding, termed H-embedding, and construct a hypernet framework under its guidance to learn task-conditioned model weights for CL tasks. Specifically, H-embedding is derived from an information theoretic measure of transferability and is designed to be online and easy to compute. Our method is also characterized by notable practicality, requiring only the storage of a low-dimensional task embedding per task and supporting efficient end-to-end training. Extensive evaluations on benchmarks including CIFAR-100, ImageNet-R, and DomainNet show that our framework performs prominently compared to various baseline and SOTA approaches, demonstrating strong potential in capturing and utilizing intrinsic task relationships. Our code is publicly available at https://github.com/viki760/Hembedding_Guided_Hypernet.

翻译：持续学习（CL）已成为当代深度神经网络应用中的关键课题，其中更高水平的前向与后向迁移对于实现有效的CL性能至关重要。现有的CL策略主要聚焦于任务模型，或通过正则化模型更新，或通过分离任务特定组件与共享组件，却往往忽视了利用任务间关联以增强迁移的潜力。为填补这一空白，我们提出一种可迁移性感知的任务嵌入（称为H-embedding），并在其指导下构建超网络框架，以学习面向CL任务的任务条件化模型权重。具体而言，H-embedding源自信息论视角的可迁移性度量，其设计支持在线计算且易于实现。本方法还具有显著的实用性，仅需为每个任务存储低维任务嵌入，并支持高效的端到端训练。在CIFAR-100、ImageNet-R和DomainNet等基准数据集上的大量实验表明，相较于各类基线方法与前沿技术，本框架均表现出突出性能，展现了在捕获与利用内在任务关联方面的强大潜力。代码已公开于https://github.com/viki760/Hembedding_Guided_Hypernet。