Learning approximations to smooth target functions of many variables from finite sets of pointwise samples is an important task in scientific computing and its many applications in computational science and engineering. Despite well over half a century of research on high-dimensional approximation, this remains a challenging problem. Yet, significant advances have been made in the last decade towards efficient methods for doing this, commencing with so-called sparse polynomial approximation methods and continuing most recently with methods based on Deep Neural Networks (DNNs). In tandem, there have been substantial advances in the relevant approximation theory and analysis of these techniques. In this work, we survey this recent progress. We describe the contemporary motivations for this problem, which stem from parametric models and computational uncertainty quantification; the relevant function classes, namely, classes of infinite-dimensional, Banach-valued, holomorphic functions; fundamental limits of learnability from finite data for these classes; and finally, sparse polynomial and DNN methods for efficiently learning such functions from finite data. For the latter, there is currently a significant gap between the approximation theory of DNNs and the practical performance of deep learning. Aiming to narrow this gap, we develop the topic of practical existence theory, which asserts the existence of dimension-independent DNN architectures and training strategies that achieve provably near-optimal generalization errors in terms of the amount of training data.

翻译：从有限点采样集合中学习多变量光滑目标函数的逼近，是科学计算及其在计算科学与工程中众多应用的重要任务。尽管对高维逼近的研究已逾半个世纪，这仍是一个具有挑战性的问题。然而，过去十年间，在实现这一目标的高效方法方面取得了显著进展，始于所谓的稀疏多项式逼近方法，并延续至近期基于深度神经网络（DNN）的方法。与此同时，这些技术的相关逼近理论与分析也取得了实质性进展。本文综述了这些近期成果。我们阐述了该问题的当代动机，这些动机源于参数化模型与计算不确定性量化；相关的函数类，即无穷维、Banach值、全纯函数类；这些函数类在有限数据下可学习性的基本限制；以及最后，用于从有限数据中高效学习此类函数的稀疏多项式和DNN方法。针对后者，目前DNN的逼近理论与深度学习的实际性能之间存在显著差距。为缩小这一差距，我们发展了实践存在性理论这一主题，该理论断言存在维度无关的DNN架构与训练策略，能在训练数据量方面实现可证明的接近最优的泛化误差。