Discovering a suitable neural network architecture for modeling complex dynamical systems poses a formidable challenge, often involving extensive trial and error and navigation through a high-dimensional hyper-parameter space. In this paper, we discuss a systematic approach to constructing neural architectures for modeling a subclass of dynamical systems, namely, Linear Time-Invariant (LTI) systems. We use a variant of continuous-time neural networks in which the output of each neuron evolves continuously as a solution of a first-order or second-order Ordinary Differential Equation (ODE). Instead of deriving the network architecture and parameters from data, we propose a gradient-free algorithm to compute sparse architecture and network parameters directly from the given LTI system, leveraging its properties. We bring forth a novel neural architecture paradigm featuring horizontal hidden layers and provide insights into why employing conventional neural architectures with vertical hidden layers may not be favorable. We also provide an upper bound on the numerical errors of our neural networks. Finally, we demonstrate the high accuracy of our constructed networks on three numerical examples.

翻译：为复杂动力系统建模寻找合适的神经网络架构是一项艰巨挑战，通常涉及大量试错及在高维超参数空间中的探索。本文讨论了一种系统化构建神经架构的方法，用于对动力系统子类——即线性时不变（LTI）系统——进行建模。我们采用连续时间神经网络的变体，其中每个神经元的输出作为一阶或二阶常微分方程（ODE）的解连续演化。不同于从数据中推导网络架构和参数，我们提出一种无梯度算法，直接利用给定LTI系统的性质计算稀疏架构和网络参数。我们提出了一种具有水平隐藏层的新型神经架构范式，并阐释了为何采用含垂直隐藏层的传统神经架构可能并不理想。此外，我们给出了所构建神经网络数值误差的上界。最后，通过三个数值算例展示了所构建网络的高精度性能。