Despite the significant advances achieved in Artificial Neural Networks (ANNs), their design process remains notoriously tedious, depending primarily on intuition, experience and trial-and-error. This human-dependent process is often time-consuming and prone to errors. Furthermore, the models are generally bound to their training contexts, with no considerations to their surrounding environments. Continual adaptiveness and automation of neural networks is of paramount importance to several domains where model accessibility is limited after deployment (e.g IoT devices, self-driving vehicles, etc.). Additionally, even accessible models require frequent maintenance post-deployment to overcome issues such as Concept/Data Drift, which can be cumbersome and restrictive. By leveraging and combining approaches from Neural Architecture Search (NAS) and Continual Learning (CL), more robust and adaptive agents can be developed. This study conducts the first extensive review on the intersection between NAS and CL, formalizing the prospective Continually-Adaptive Neural Networks (CANNs) paradigm and outlining research directions for lifelong autonomous ANNs.

翻译：尽管人工神经网络（ANN）取得了显著进展，但其设计过程仍然以直觉、经验和试错为主，这一依赖人工的过程耗时且易出错。此外，模型通常局限于其训练环境，未考虑周围环境。神经网络的持续适应性与自动化对模型在部署后访问受限的领域（如物联网设备、自动驾驶车辆等）至关重要。即使可访问的模型也需在部署后进行频繁维护以应对概念/数据漂移等问题，这一过程可能繁琐且具有限制性。通过利用和结合神经架构搜索（NAS）与持续学习（CL）的方法，可以开发出更鲁棒、更具适应性的智能体。本研究首次对NAS与CL的交叉领域进行广泛综述，正式提出前瞻性的持续自适应神经网络（CANNs）范式，并概述了实现终身自主ANN的研究方向。