Neural architecture search (NAS), which automates the architectural design process of deep neural networks (DNN), has attracted increasing attention. Multimodal DNNs that necessitate feature fusion from multiple modalities benefit from NAS due to their structural complexity; however, constructing an architecture for multimodal DNNs through NAS requires a substantial amount of labeled training data. Thus, this paper proposes a self-supervised learning (SSL) method for architecture search of multimodal DNNs. The proposed method applies SSL comprehensively for both the architecture search and model pretraining processes. Experimental results demonstrated that the proposed method successfully designed architectures for DNNs from unlabeled training data.

翻译：神经架构搜索（NAS）通过自动化深度神经网络（DNN）的架构设计过程，已引起越来越多的关注。多模态DNN需要融合来自多种模态的特征，其结构复杂性使得它们能从NAS中获益；然而，通过NAS为多模态DNN构建架构需要大量带标签的训练数据。因此，本文提出了一种用于多模态DNN架构搜索的自监督学习（SSL）方法。所提方法将SSL全面应用于架构搜索和模型预训练两个过程。实验结果表明，所提方法成功地从无标签训练数据中为DNN设计了架构。