This work introduces Neural Chronos Ordinary Differential Equations (Neural CODE), a deep neural network architecture that fits a continuous-time ODE dynamics for predicting the chronology of a system both forward and backward in time. To train the model, we solve the ODE as an initial value problem and a final value problem, similar to Neural ODEs. We also explore two approaches to combining Neural CODE with Recurrent Neural Networks by replacing Neural ODE with Neural CODE (CODE-RNN), and incorporating a bidirectional RNN for full information flow in both time directions (CODE-BiRNN), and variants with other update cells namely GRU and LSTM: CODE-GRU, CODE-BiGRU, CODE-LSTM, CODE-BiLSTM. Experimental results demonstrate that Neural CODE outperforms Neural ODE in learning the dynamics of a spiral forward and backward in time, even with sparser data. We also compare the performance of CODE-RNN/-GRU/-LSTM and CODE-BiRNN/-BiGRU/-BiLSTM against ODE-RNN/-GRU/-LSTM on three real-life time series data tasks: imputation of missing data for lower and higher dimensional data, and forward and backward extrapolation with shorter and longer time horizons. Our findings show that the proposed architectures converge faster, with CODE-BiRNN/-BiGRU/-BiLSTM consistently outperforming the other architectures on all tasks.

翻译：本文提出了神经时间常微分方程（Neural Chronos ODE，简称Neural CODE），一种通过拟合连续时间ODE动态来预测系统时间前后演变的深度神经网络架构。为训练该模型，我们类似于神经ODE方法，将ODE分别作为初值问题和终值问题进行求解。我们还探索了两种将Neural CODE与循环神经网络结合的方法：用Neural CODE替代Neural ODE（CODE-RNN），以及引入双向RNN实现两个时间方向的完整信息流（CODE-BiRNN），并包含采用其他更新单元（GRU和LSTM）的变体：CODE-GRU、CODE-BiGRU、CODE-LSTM、CODE-BiLSTM。实验结果表明，即使在数据稀疏的情况下，Neural CODE在习得螺旋线时间正向与反向动态方面仍优于Neural ODE。我们还在三个真实时间序列数据任务上对比了CODE-RNN/-GRU/-LSTM和CODE-BiRNN/-BiGRU/-BiLSTM与ODE-RNN/-GRU/-LSTM的性能：低维与高维数据的缺失值插补，以及短时域与长时域的正向与反向外推。研究结果表明，所提出的架构收敛速度更快，其中CODE-BiRNN/-BiGRU/-BiLSTM在所有任务上均始终优于其他架构。