We show how continuous-depth neural ODE models can be framed as single-layer, infinite-width nets using the Chen--Fliess series expansion for nonlinear ODEs. In this net, the output ''weights'' are taken from the signature of the control input -- a tool used to represent infinite-dimensional paths as a sequence of tensors -- which comprises iterated integrals of the control input over a simplex. The ''features'' are taken to be iterated Lie derivatives of the output function with respect to the vector fields in the controlled ODE model. The main result of this work applies this framework to derive compact expressions for the Rademacher complexity of ODE models that map an initial condition to a scalar output at some terminal time. The result leverages the straightforward analysis afforded by single-layer architectures. We conclude with some examples instantiating the bound for some specific systems and discuss potential follow-up work.
翻译:我们展示了如何利用非线性ODE的陈-弗里斯级数展开,将连续深度神经ODE模型构建为单层无限宽网络。在该网络中,输出"权重"取自控制输入的签名——一种将无限维路径表示为张量序列的工具,其包含控制输入在单纯形上的迭代积分。而"特征"则选自受控ODE模型中输出函数关于向量场的迭代李导数。本文主要成果在于应用该框架,推导了将初始条件映射至终端时刻标量输出的ODE模型的拉德马赫复杂度紧凑表达式。该结果充分利用了单层架构的简洁分析优势。最后,我们通过若干具体系统实例给出复杂度上界,并探讨了后续可能的研究方向。