We study the approximation of multivariate functions with tensor networks (TNs). The main conclusion of this work is an answer to the following two questions: ``What are the approximation capabilities of TNs?" and "What is an appropriate model class of functions that can be approximated with TNs?" To answer the former, we show that TNs can (near to) optimally replicate $h$-uniform and $h$-adaptive approximation, for any smoothness order of the target function. Tensor networks thus exhibit universal expressivity w.r.t. isotropic, anisotropic and mixed smoothness spaces that is comparable with more general neural networks families such as deep rectified linear unit (ReLU) networks. Put differently, TNs have the capacity to (near to) optimally approximate many function classes -- without being adapted to the particular class in question. To answer the latter, as a candidate model class we consider approximation classes of TNs and show that these are (quasi-)Banach spaces, that many types of classical smoothness spaces are continuously embedded into said approximation classes and that TN approximation classes are themselves not embedded in any classical smoothness space.

翻译：我们研究使用张量网络逼近多元函数的问题。本工作的主要结论为以下两个问题提供了答案：“张量网络具有何种逼近能力？”以及“何种函数模型类适合用张量网络逼近？”。针对前者，我们证明对于目标函数的任意光滑阶，张量网络能够（近似）最优地复现$h$-均匀与$h$-自适应逼近。因此，张量网络对各项同性、各向异性及混合光滑空间具有普遍表达能力，其表现可与更广义的神经网络族（如深度整流线性单元网络）相媲美。换言之，张量网络具备（近似）最优逼近多种函数类的能力——而无需针对特定函数类进行调整。针对后者，我们考虑将张量网络的逼近类作为候选模型类，证明这些逼近类构成（拟）巴拿赫空间，多种经典光滑空间连续嵌入到所述逼近类中，且张量网络逼近类本身不嵌入任何经典光滑空间。