In our work, we build upon the established connection between Residual Neural Networks (ResNets) and continuous-time control systems known as NeurODEs. By construction, NeurODEs have been limited to constant-width layers, making them unsuitable for modeling deep learning architectures with width-varying layers. In this paper, we propose a continuous-time Autoencoder, which we call AutoencODE, and we extend to this case the mean-field control framework already developed for usual NeurODEs. In this setting, we tackle the case of low Tikhonov regularization, resulting in potentially non-convex cost landscapes. While the global results obtained for high Tikhonov regularization may not hold globally, we show that many of them can be recovered in regions where the loss function is locally convex. Inspired by our theoretical findings, we develop a training method tailored to this specific type of Autoencoders with residual connections, and we validate our approach through numerical experiments conducted on various examples.

翻译：在我们的工作中，我们基于残差神经网络与连续时间控制系统（即神经ODE）之间已有的联系展开研究。由于构造上的限制，神经ODE仅限于恒定宽度的层，因此无法对具有变宽度层的深度学习架构进行建模。本文提出了一种连续时间自编码器，称之为自编码ODE，并将已为常规神经ODE开发的均值场控制框架拓展至该情形。在此框架下，我们研究了低Tikhonov正则化的情况，这可能导致非凸的代价景观。尽管高Tikhonov正则化下获得的全局结果可能无法全局成立，但我们表明，在损失函数局部凸的区域中，许多结果可以恢复。受理论发现的启发，我们针对这类具有残差连接的自编码器开发了一种专门训练方法，并通过多种示例上的数值实验验证了该方法的有效性。