With the rapid development of deep learning in various fields of science and technology, such as speech recognition, image classification, and natural language processing, recently it is also widely applied in the functional data analysis (FDA) with some empirical success. However, due to the infinite dimensional input, we need a powerful dimension reduction method for functional learning tasks, especially for the nonlinear functional regression. In this paper, based on the idea of smooth kernel integral transformation, we propose a functional deep neural network with an efficient and fully data-dependent dimension reduction method. The architecture of our functional net consists of a kernel embedding step: an integral transformation with a data-dependent smooth kernel; a projection step: a dimension reduction by projection with eigenfunction basis based on the embedding kernel; and finally an expressive deep ReLU neural network for the prediction. The utilization of smooth kernel embedding enables our functional net to be discretization invariant, efficient, and robust to noisy observations, capable of utilizing information in both input functions and responses data, and have a low requirement on the number of discrete points for an unimpaired generalization performance. We conduct theoretical analysis including approximation error and generalization error analysis, and numerical simulations to verify these advantages of our functional net.

翻译：随着深度学习在语音识别、图像分类、自然语言处理等科学与技术领域的快速发展，近年来该方法也被广泛应用于函数型数据分析（FDA）并取得了一定的实证成功。然而，由于输入空间的无限维特性，我们需要为函数型学习任务，特别是非线性函数回归问题，设计强大的降维方法。本文基于光滑核积分变换的思想，提出了一种具有高效且完全数据驱动降维方法的函数深度神经网络。该函数网络架构包含三个步骤：核嵌入步骤（利用数据依赖的光滑核进行积分变换）、投影步骤（基于嵌入核的特征函数基通过投影实现降维），以及最终用于预测的深度ReLU神经网络。光滑核嵌入的应用使我们的函数网络具有离散化不变性、高效性以及对噪声观测的鲁棒性，能够同时利用输入函数和响应数据中的信息，且在保持完整泛化性能的前提下对离散点数量要求较低。我们通过包括逼近误差和泛化误差分析在内的理论分析以及数值模拟，验证了所提函数网络的上述优势。