Autonomous systems (AS) often use Deep Neural Network (DNN) classifiers to allow them to operate in complex, high-dimensional, non-linear, and dynamically changing environments. Due to the complexity of these environments, DNN classifiers may output misclassifications during operation when they face domains not identified during development. Removing a system from operation for retraining becomes impractical as the number of such AS increases. To increase AS reliability and overcome this limitation, DNN classifiers need to have the ability to adapt during operation when faced with different operational domains using a few samples (e.g. 2 to 100 samples). However, retraining DNNs on a few samples is known to cause catastrophic forgetting and poor generalisation. In this paper, we introduce Dynamic Incremental Regularised Adaptation (DIRA), an approach for dynamic operational domain adaption of DNNs using regularisation techniques. We show that DIRA improves on the problem of forgetting and achieves strong gains in performance when retraining using a few samples from the target domain. Our approach shows improvements on different image classification benchmarks aimed at evaluating robustness to distribution shifts (e.g.CIFAR-10C/100C, ImageNet-C), and produces state-of-the-art performance in comparison with other methods from the literature.

翻译：自主系统（AS）常使用深度神经网络（DNN）分类器，以在复杂、高维、非线性且动态变化的环境中运行。由于这些环境的复杂性，DNN分类器在遇到开发阶段未识别的领域时，可能在运行过程中输出错误分类。随着自主系统数量的增加，将系统从运行中撤下进行重新训练变得不现实。为提升自主系统可靠性并克服这一限制，DNN分类器需具备在运行时面对不同操作域时，仅用少量样本（例如2至100个样本）进行自适应调整的能力。然而，基于少量样本重新训练DNN会导致灾难性遗忘和泛化能力差的问题。本文提出动态增量正则化自适应（DIRA），一种利用正则化技术实现DNN运行域动态自适应的新方法。我们证明，DIRA可改善遗忘问题，并在使用目标域少量样本重新训练时实现显著的性能提升。该方法在多个旨在评估分布偏移鲁棒性的图像分类基准测试（如CIFAR-10C/100C、ImageNet-C）上均表现出改进效果，与文献中的其他方法相比，达到了最优性能。