We introduce a method for flexible and efficient continual learning in open-vocabulary image classification, drawing inspiration from the complementary learning systems observed in human cognition. Specifically, we propose to combine predictions from a CLIP zero-shot model and the exemplar-based model, using the zero-shot estimated probability that a sample's class is within the exemplar classes. We also propose a "tree probe" method, an adaption of lazy learning principles, which enables fast learning from new examples with competitive accuracy to batch-trained linear models. We test in data incremental, class incremental, and task incremental settings, as well as ability to perform flexible inference on varying subsets of zero-shot and learned categories. Our proposed method achieves a good balance of learning speed, target task effectiveness, and zero-shot effectiveness. Code will be available at https://github.com/jessemelpolio/TreeProbe.

翻译：我们提出了一种灵活且高效的开放词汇图像分类持续学习方法，该方法借鉴了人类认知中的互补学习系统机制。具体而言，我们提出将CLIP零样本模型与基于样本的模型预测相结合，利用零样本估计样本类别属于样本类别的概率。我们还提出了一种"树探针"方法，该方法是对惰性学习原理的改进，能够从新样本中快速学习，同时其准确率可与批量训练的线性模型相竞争。我们在数据增量、类别增量和任务增量设置下进行了测试，并验证了在零样本类别与已学习类别的不同子集上进行灵活推理的能力。我们提出的方法在学习速度、目标任务效果和零样本效果之间实现了良好平衡。代码将在https://github.com/jessemelpolio/TreeProbe 提供。