Meta-Continual Learning (Meta-CL) enables models to learn new classes from limited labelled samples, making it promising for IoT applications where manual labelling is costly. However, existing studies focus on accuracy while ignoring deployment viability on resource-constrained hardware. Thus, we present MetaCLBench, a benchmark framework that evaluates Meta-CL methods for both accuracy and deployment-critical metrics (memory footprint, latency, and energy consumption) on real IoT devices with RAM sizes ranging from 512 MB to 4 GB. We evaluate six Meta-CL methods across three architectures (CNN, YAMNet, ViT) and five datasets spanning image and audio modalities. Our evaluation reveals that, depending on the dataset, up to three of six methods cause out-of-memory failures on sub-1 GB devices, significantly narrowing viable deployment options. LifeLearner achieves near-oracle accuracy while consuming 2.54-7.43x less energy than the Oracle method. Notably, larger or more sophisticated architectures such as ViT and YAMNet do not necessarily yield better Meta-CL performance, with results varying across datasets and modalities, challenging conventional assumptions about model complexity. Finally, we provide practical deployment guidelines and will release our framework upon publication to enable fair evaluation across both accuracy and system-level metrics.

翻译：元持续学习（Meta-CL）使模型能够从有限的标注样本中学习新类别，这使其在人工标注成本高昂的物联网应用中具有广阔前景。然而，现有研究主要关注准确性，而忽视了在资源受限硬件上的部署可行性。为此，我们提出了MetaCLBench，这是一个基准框架，用于在内存大小从512 MB到4 GB的真实物联网设备上，评估元持续学习方法在准确性及部署关键指标（内存占用、延迟和能耗）上的表现。我们在三种架构（CNN、YAMNet、ViT）和涵盖图像与音频模态的五个数据集上评估了六种元持续学习方法。评估结果显示，根据数据集的不同，六种方法中最多有三种会在内存小于1 GB的设备上导致内存不足故障，从而显著缩小了可行的部署选项。LifeLearner在实现接近Oracle准确性的同时，能耗比Oracle方法低2.54至7.43倍。值得注意的是，更大或更复杂的架构（如ViT和YAMNet）并不一定能带来更好的元持续学习性能，其结果因数据集和模态而异，这对关于模型复杂性的传统假设提出了挑战。最后，我们提供了实用的部署指南，并将在论文发表后开源我们的框架，以支持在准确性和系统级指标上进行公平评估。