We investigate the expressive power of deep residual neural networks idealized as continuous dynamical systems through control theory. Specifically, we consider two properties that arise from supervised learning, namely universal interpolation - the ability to match arbitrary input and target training samples - and the closely related notion of universal approximation - the ability to approximate input-target functional relationships via flow maps. Under the assumption of affine invariance of the control family, we give a characterisation of universal interpolation, showing that it holds for essentially any architecture with non-linearity. Furthermore, we elucidate the relationship between universal interpolation and universal approximation in the context of general control systems, showing that the two properties cannot be deduced from each other. At the same time, we identify conditions on the control family and the target function that ensures the equivalence of the two notions.

翻译：我们通过控制理论将深度残差神经网络理想化为连续动力系统，探究其表达能力。具体而言，我们关注监督学习引发的两个性质：通用内插能力（即匹配任意输入与目标训练样本的能力）及其密切相关的通用逼近能力（即通过流映射逼近输入-目标函数关系的能力）。在控制族满足仿射不变性的假设下，我们给出了通用内插的表征条件，表明该性质本质上适用于任意含非线性元素的架构。此外，我们阐明了通用内插与通用逼近在一般控制系统框架下的关联，证明这两个性质无法相互推导。同时，我们识别出确保两种概念等价的控制族与目标函数条件。