The importance of proper data normalization for deep neural networks is well known. However, in continuous-time state-space model estimation, it has been observed that improper normalization of either the hidden state or hidden state derivative of the model estimate, or even of the time interval can lead to numerical and optimization challenges with deep learning based methods. This results in a reduced model quality. In this contribution, we show that these three normalization tasks are inherently coupled. Due to the existence of this coupling, we propose a solution to all three normalization challenges by introducing a normalization constant at the state derivative level. We show that the appropriate choice of the normalization constant is related to the dynamics of the to-be-identified system and we derive multiple methods of obtaining an effective normalization constant. We compare and discuss all the normalization strategies on a benchmark problem based on experimental data from a cascaded tanks system and compare our results with other methods of the identification literature.

翻译：深度神经网络中适当的数据归一化的重要性是众所周知的。然而，在连续时间状态空间模型估计中，人们发现对模型估计的隐藏状态或隐藏状态导数，甚至时间间隔的不当归一化，会导致基于深度学习的方法出现数值和优化挑战，从而降低模型质量。在本文中，我们展示了这三项归一化任务本质上是耦合的。由于这种耦合的存在，我们通过在状态导数层面引入一个归一化常数，提出了解决所有三个归一化挑战的方案。我们表明，归一化常数的适当选择与待辨识系统的动力学相关，并推导出了获取有效归一化常数的多种方法。我们基于来自级联水箱系统的实验数据，在一个基准问题上比较并讨论了所有归一化策略，并将我们的结果与辨识文献中的其他方法进行了对比。