We propose a novel, brain-inspired deep neural network model known as the Deep Oscillatory Neural Network (DONN). Deep neural networks like the Recurrent Neural Networks indeed possess sequence processing capabilities but the internal states of the network are not designed to exhibit brain-like oscillatory activity. With this motivation, the DONN is designed to have oscillatory internal dynamics. Neurons of the DONN are either nonlinear neural oscillators or traditional neurons with sigmoidal or ReLU activation. The neural oscillator used in the model is the Hopf oscillator, with the dynamics described in the complex domain. Input can be presented to the neural oscillator in three possible modes. The sigmoid and ReLU neurons also use complex-valued extensions. All the weight stages are also complex-valued. Training follows the general principle of weight change by minimizing the output error and therefore has an overall resemblance to complex backpropagation. A generalization of DONN to convolutional networks known as the Oscillatory Convolutional Neural Network is also proposed. The two proposed oscillatory networks are applied to a variety of benchmark problems in signal and image/video processing. The performance of the proposed models is either comparable or superior to published results on the same data sets.

翻译：我们提出了一种新颖的、受大脑启发的深度神经网络模型——深度振荡神经网络（DONN）。虽然循环神经网络等深度神经网络具备序列处理能力，但其内部状态设计并未展现出类似大脑的振荡活动。基于此动机，DONN被设计为具有振荡内部动力学特性。DONN的神经元要么是非线性神经振荡器，要么是采用Sigmoid或ReLU激活函数的传统神经元。模型中使用的神经振荡器为Hopf振荡器，其动力学过程在复数域中描述。输入可以三种模式呈现给神经振荡器。Sigmoid和ReLU神经元也采用复数域扩展。所有权重阶段同样为复数值。训练遵循通过最小化输出误差调整权重的通用原则，因此在整体上与复数反向传播类似。本文还提出了DONN向卷积网络的推广形式——振荡卷积神经网络。两种提出的振荡网络被应用于信号与图像/视频处理领域的多种基准测试问题。所提出模型在相同数据集上的性能与已有文献结果相当或更优。